Eyewear devices with focus tunable lenses

ABSTRACT

An eyewear device comprises a left lens assembly and a right lens assembly. The left lens assembly includes a left focus tunable lens and a left focus fixed lens. A right lens assembly includes a right focus tunable lens and a right focus fixed lens. The eyewear device may be used in 3D display applications, virtual reality applications, augmented reality applications, remote presence applications, etc. The eyewear device may also be used as vision correction glasses.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application claims priority to U.S. Provisional Patent ApplicationNo. 62/414,901, filed on Oct. 31, 2016.

TECHNOLOGY

The present invention relates generally to eyewear devices, and inparticular, to eyewear devices with focus tunable lenses inaccommodation-vergence solutions.

BACKGROUND

Retina refers to a substantial part of an interior surface of an eye(e.g., of a human, of a viewer, etc.) with visual sensors opposite tothe pupil of the eye. Fovea refers to a relatively tiny middle portionof the retina that hosts numerous visual sensors capable of the sharpestvision and the most sensitive color reception in the eye.

The human brain uses a vergence process to control extraocular musclesto simultaneously converge or diverge two eyes of the (human) viewertoward any visible object in a scene in order to support the perceptionof the object as a 3D object. At the same time, the human brain uses anaccommodation process to control ciliary muscles to adapt each of eyelenses located behind pupils in the two eyes to certain focal lengths(or powers) in order to support the clear vision or foveal vision of theobject.

When viewing real world objects in a real world environment (or scene),the human brain uses natural inter-dependent accommodation and vergenceprocesses to simultaneously control the ciliary muscles and theextraocular muscles to adapt both focal lengths of the viewer's separateeye lenses in order to support the clear vision (or the foveal vision)of a real world object located at a certain spatial position andconcurrently to converge and diverge both of the eyes toward the realworld object at the certain spatial position in order to support theperception of the real world environment that includes the real worldobject.

In contrast, when viewing 3D images with near-eye displays, the humanbrain has to go through a relearning process to use conflictingaccommodation and vergence processes to control the ciliary muscles andthe extraocular muscles. These conflicting accommodation and vergenceprocesses control the ciliary muscles and the extraocular muscles inviewing 3D images very differently from how the accommodation andvergence processes control the ciliary muscles and the extraocularmuscles in viewing real world objects in a real world environment.

More specifically, the human brain needs to control the ciliary musclesto set the eye lenses of the viewer's eyes to a constant focal length inorder to support the clear vision (or foveal vision) of images renderedon the near-eye displays located at a fixed distance from the eyes,regardless of where a depicted object in the images that is being viewedby the viewer is supposed to be located. At the same time when theciliary muscles fix the focal lengths of the eye lenses to view clearlyat the near-eye displays, the human brain still needs to control theextraocular muscles to converge or diverge the eyes simultaneouslytoward the depicted object in the images at a distance away from thenear-eye displays in order to support the perception of the object as a3D object.

This is known as the accommodation-vergence conflict. That is, the brainhas to control the ciliary and extraocular muscles very differently inviewing 3D images than in viewing real world objects. Unfortunately, theaccommodation-vergence conflict in 3D image viewing can cause frequentand serious physiological discomforts/sickness such as nauseas,headaches, disorientation, etc., during and after viewing 3D images.

The approaches described in this section are approaches that could bepursued, but not necessarily approaches that have been previouslyconceived or pursued. Therefore, unless otherwise indicated, it shouldnot be assumed that any of the approaches described in this sectionqualify as prior art merely by virtue of their inclusion in thissection. Similarly, issues identified with respect to one or moreapproaches should not assume to have been recognized in any prior art onthe basis of this section, unless otherwise indicated.

BRIEF DESCRIPTION OF DRAWINGS

The present invention is illustrated by way of example, and not by wayof limitation, in the figures of the accompanying drawings and in whichlike reference numerals refer to similar elements and in which:

FIG. 1A illustrates a cross-sectional view of an example human eye; FIG.1B illustrates example viewing of a real world object located in anobject plane by left eye and right eye;

FIG. 2A, FIG. 2B and FIG. 2D illustrate example viewing of virtualobject(s) depicted in a stereoscopic image comprising a left image and aright image; FIG. 2C illustrates example tracking of a viewer's vergenceangles in viewing a time sequence of stereoscopic images; FIG. 2Eillustrates an example blurring filter;

FIG. 3A through FIG. 3C illustrate example video streaming servers andclients;

FIG. 4A and FIG. 4B illustrate example process flows;

FIG. 5 illustrates an example hardware platform on which a computer or acomputing device as described herein may be implemented;

FIG. 6A through FIG. 6E illustrate example vision fields of an eye;

FIG. 7A and FIG. 7B illustrate example focus tunable lenses; and

FIG. 8A through FIG. 8F illustrate example eyewear devices.

DESCRIPTION OF EXAMPLE EMBODIMENTS

Example embodiments, which relate to eyewear devices with focus tunablelenses in accommodation-vergence solutions, are described herein. In thefollowing description, for the purposes of explanation, numerousspecific details are set forth in order to provide a thoroughunderstanding of the present invention. It will be apparent, however,that the present invention may be practiced without these specificdetails. In other instances, well-known structures and devices are notdescribed in exhaustive detail, in order to avoid unnecessarilyoccluding, obscuring, or obfuscating the present invention.

Example embodiments are described herein according to the followingoutline:

-   -   1. GENERAL OVERVIEW    -   2. ACCOMMODATION AND VERGENCE    -   3. CONFLICT BETWEEN ACCOMMODATION AND VERGENCE    -   4. SOLVING CONFLICT BETWEEN ACCOMMODATION AND VERGENCE    -   5. TRACKING VERGENCE ANGLES    -   6. VISION FIELDS    -   7. EXAMPLE FOCUS TUNABLE LENSES    -   8. EXAMPLE EYEWEAR DEVICES    -   9. EXAMPLE VIDEO STREAMING SERVERS AND CLIENTS    -   10. EXAMPLE PROCESS FLOWS    -   11. IMPLEMENTATION MECHANISMS—HARDWARE OVERVIEW    -   12. EQUIVALENTS, EXTENSIONS, ALTERNATIVES AND MISCELLANEOUS

1. General Overview

This overview presents a basic description of some aspects of an exampleembodiment of the present invention. It should be noted that thisoverview is not an extensive or exhaustive summary of aspects of theexample embodiment. Moreover, it should be noted that this overview isnot intended to be understood as identifying any particularlysignificant aspects or elements of the example embodiment, nor asdelineating any scope of the example embodiment in particular, nor theinvention in general. This overview merely presents some concepts thatrelate to the example embodiment in a condensed and simplified format,and should be understood as merely a conceptual prelude to a moredetailed description of example embodiments that follows below. Notethat, although separate embodiments are discussed herein, anycombination of embodiments and/or partial embodiments discussed hereinmay be combined to form further embodiments.

Example embodiments described herein relate to eyewear devices. Aneyewear device comprises a left lens assembly and a right lens assembly.The left lens assembly includes a left focus tunable lens and a leftfocus fixed lens. A right lens assembly includes a right focus tunablelens and a right focus fixed lens. The eyewear device may be used in 3Ddisplay applications, virtual reality applications, augmented realityapplications, remote presence applications, etc. The eyewear device mayalso be used as vision correction glasses.

In some embodiments, a vision device comprises one or more imagedisplays that display a left image and a right image of a stereoscopicimage; an eyewear device as described herein; the eyewear deviceprojects the left image and the right image to a virtual object depthdepending on a viewer's vergence angles. In some embodiments, the visiondevice further comprises one or more gaze tracking devices that trackand determine the viewer's vergence angles at runtime.

Example embodiments described herein relate to solvingaccommodation-vergence conflicts in rendering and viewing 3D images (ormulti-view images) through auto-tunable lenses. One or more gazetracking devices are used to track a virtual object depth to which aviewer's left eye and the viewer's right eye are directed. Astereoscopic image comprising a left image and a right image arerendered on one or more image displays. The left image is projected to avirtual object plane at a virtual object depth with a left lens assemblyof an eyewear device. The right image is projected to the virtual objectplane at the virtual object depth with a right lens assembly of theeyewear device. The left lens assembly comprises a left focus tunablelens and a left focus fixed lens, whereas the right lens assemblycomprises a right focus tunable lens and a right focus fixed lens.

In some example embodiments, mechanisms as described herein form a partof a media processing system, including but not limited to any of:near-eye displays, cloud-based server, mobile device, virtual realitysystem, augmented reality system, remote presence system, head updisplay device, helmet mounted display device, zSpace displays,CAVE-type system or wall-sized display, video game device, displaydevice, media player, media server, media production system, camerasystems, home-based systems, communication devices, video processingsystem, video codec system, studio system, streaming server, cloud-basedcontent service system, a handheld device, game machine, television,cinema display, laptop computer, netbook computer, tablet computer,cellular radiotelephone, electronic book reader, point of sale terminal,desktop computer, computer workstation, computer server, computer kiosk,or various other kinds of terminals and media processing units.

Various modifications to the preferred embodiments and the genericprinciples and features described herein will be readily apparent tothose skilled in the art. Thus, the disclosure is not intended to belimited to the embodiments shown, but is to be accorded the widest scopeconsistent with the principles and features described herein.

2. Accommodation and Vergence

FIG. 1A illustrates a cross-sectional view of an example human eye 100of a viewer, as viewed from directly above the viewer's head. Asillustrated, the eye (100) has an optical axis 102 (vertical to alateral line 112) that passes through the center point of a pupil 104located in the front portion of the eye (100) and the center point of afovea 106 located in a retina 110 in the back portion of the eye (100).Light collected through an eye lens 108 located behind the pupil (104)may be projected by the eye lens (108) onto the fovea (106). For thepurpose of illustration only, the eye lens (108) may be opticallycharacterized with an eye focal length. As the eye lens may or may notrepresent an optical lens of a single focal length, the eye focal lengthas described herein may refer to one of: a focal length for light nearthe optical axis (102), an average focal length of a central portion ofthe eye lens, an effective focal length for light near the optical axis(102), a focal length in reference to light projected onto the fovea, afocal length of a locally near perfect lens relative to the fovealvision, etc. It should be noted that in various embodiments, the eyelens may be modeled as a single lens or multiple lenses, one, some orall of which may have variable focal lenses controllable for examplethrough ciliary muscles in or associated with the eye (100).

FIG. 1B illustrates example viewing of a real world object 114 locatedin an object plane 116 by a viewer's left eye 100-1 and right eye 100-2.As illustrated, the object plane (116)—in which the real world object(114) is located—is perpendicular to the viewer's frontal viewingdirection 118, and is parallel to the viewer's inter-pupillary line 120.

To effectuate a clear vision of the real world object (114) through bothof the left fovea (106-1) of the left eye (100-1) and the right fovea(106-2) of the right eye (100-2), the viewer's brain uses a vergenceprocess (e.g., a divergence process, a convergence process, etc.) tocontrol extraocular muscles in both eyes simultaneously to orient theleft eye (100-1) and the right eye (100-2) toward the real world object(114). If the viewer's optical axes (102-1 and 102-2, vertical tolateral lines 112-1 and 112-2 respectively) were previously directed ata spatial point closer than the real world object (114), the viewer'sbrain uses a divergence process to control the extraocular muscles inboth eyes to diverge simultaneously the left eye (100-1) and the righteye (100-2) toward the real world object (114). Otherwise, if theviewer's optical axes (102-1 and 102-2) were previously directed at aspatial point farther than the real world object (114), the viewer'sbrain uses a convergence process to control the extraocular muscles inboth eyes to converge simultaneously the left eye (100-1) and the righteye (100-2) toward the real world object (114).

As a result, the left optical axis (102-1) of the left eye (100-1) andthe right optical axis (102-2) of the right eye (100-2) (e.g., withnormal vision) coincide at the real world object (114) to cause lightfrom the real world object (114) to be projected onto both of the leftfovea (106-1) of the left eye (100-1) and the right fovea (106-2) of theright eye (100-2).

In viewing real world objects in a real world environment/scene,accommodation and vergence processes/functions are not independent butrather inter-dependent in the viewer's brain to control muscles to verge(converge/diverge) toward an object and concurrently (e.g., adapt to)focus on the same object. For example, at the same time while thevergence process is used to control the extraocular muscles in both eyessimultaneously to orient the left eye (100-1) and the right eye (100-2)toward the real world object (114), the viewer's brain uses anaccommodation process to control ciliary muscles in both eyessimultaneously to focus the left eye (100-1) and the right eye (100-2)onto the real world object (114). The focal lengths of the left eye(100-1) and the right eye (100-2) may be adjusted by the accommodationprocess such that light (e.g., emitted, reflected, etc.) from the realworld object (114) is focused at image planes (or retinas) coincidingthe left fovea (106-1) and the right fovea (106-2). More specifically,the focal length of the left eye (100-1) may be set by the accommodationprocess based at least in part on a (left) distance 122-1 between thereal world object (114) and the left eye lens (108-1) to cause the lightfrom the real world object (114) to be focused at a left image plane (orleft retina) coinciding the left fovea (106-1), whereas the focal lengthof the right eye (100-2) may be set by the accommodation process basedat least in part on a (right) distance 122-2 between the real worldobject (114) and the right eye lens (108-2) to cause the light from thereal world object (114) to be focused at a right image plane (or rightretina) coinciding the right fovea (106-2). In scenarios in which thereal world object (114) is located from the viewer's eyes at a distancemuch (e.g., ten times, etc.) greater than the viewer's inter-pupillarydistance (the distance between the viewer's two eyes) along theinter-pupillary line (120), the focal lengths of the left eye (100-1)and the right eye (100-2) may be adjusted by the accommodation processto the same focal length or approximately same focal lengths.

3. Conflict Between Accommodation and Vergence

In some embodiments, stereoscopic images or multi-view images asdescribed herein can be captured with one or more camera systemsdeployed in one or more spatial environments. Example spatialenvironments may include, but are not limited to only, any of: physicalspatial environment, simulated spatial environment, movie studios,outdoor scenes, indoor scenes, tunnels, streets, vehicles, ships,aircrafts, outer space, etc. Example camera systems may include, but arenot limited to only, any of: 3D cameras, multi-view cameras, light fieldcameras, multiple cameras with overlapping and/or non-overlapping visionfields, digital cameras, analog cameras, webcams, etc.

A left image, a right image, an image of a specific view in a pluralityof different views, etc., of a stereoscopic image or a multi-view imageas described herein may be recorded or assembled as pixel values fordistributed pixels of an image frame.

FIG. 2A illustrates example viewing of a virtual object 214 depicted ina stereoscopic image comprising a left image 202-1 and a right image202-2. The virtual object (214) may be represented in the stereoscopicimage as located in a virtual object plane 216 by the viewer's left eye(100-1) and right eye (100-2). For the purpose of illustration only, thevirtual object plane (216)—in which the virtual object (214) is(virtually) located—is perpendicular to the viewer's frontal viewingdirection (118), and is parallel to the viewer's inter-pupillary line(120).

To effectuate a clear vision of the virtual object (214) through both ofthe left fovea (106-1) of the left eye (100-1) and the right fovea(106-2) of the right eye (100-2), the viewer's brain uses a vergenceprocess (e.g., a divergence process, a convergence process, etc.) tocontrol extraocular muscles in both eyes simultaneously to orient theleft eye (100-1) and the right eye (100-2) toward the virtual object(214). If the viewer's optical axes (102-1 and 102-2) were previouslydirected at a virtual spatial point (depicted in the stereoscopic image)closer than the virtual object (214), the viewer's brain uses adivergence process to control the extraocular muscles in both eyes todiverge simultaneously the left eye (100-1) and the right eye (100-2)toward the virtual object (214). Otherwise, if the viewer's optical axes(102-1 and 102-2) were previously directed at a virtual spatial point(depicted in the stereoscopic image) farther than the virtual object(214), the viewer's brain uses a convergence process to control theextraocular muscles in both eyes to converge simultaneously the left eye(100-1) and the right eye (100-2) toward the virtual object (214).

As a result, the left optical axis (102-1) of the left eye (100-1) andthe right optical axis (102-2) of the right eye (100-2) coincide at thevirtual object (214) to cause light from left pixels 224-1 (in the leftimage (202-1)) and right pixels 224-2 (in the right image (202-2))depicting the virtual object (214) to be respectively projected onto theleft fovea (106-1) of the left eye (100-1) and the right fovea (106-2)of the right eye (100-2).

At the same time while the vergence process is used to control theextraocular muscles in both eyes simultaneously to orient the left eye(100-1) and the right eye (100-2) toward the virtual object (214), theviewer's brain uses an accommodation process to control ciliary musclesin both eyes simultaneously to focus the left eye (100-1) and the righteye (100-2) respectively onto the left pixels (224-1) and the rightpixels (224-2) depicting the virtual object (214). The focal lengths ofthe left eye (100-1) and the right eye (100-2) may be adjusted by theaccommodation process such that light from the left pixels (224-1) andthe right pixels (224-2) depicting the virtual object (214) is focusedat respective image planes (or retinas) coinciding the left fovea(106-1) and the right fovea (106-2). More specifically, the focal lengthof the left eye (100-1) may be set by the accommodation process based atleast in part on a (left) distance 222-1 between the left pixels (224-1)and the left eye lens (108-1) to cause the light from the left pixels(224-1) to be focused at the left image plane (or left retina)coinciding the left fovea (106-1), whereas the focal length of the righteye (100-2) may be set by the accommodation process based at least inpart on a (right) distance 222-2 between the right pixels (224-2) andthe right eye lens (108-2) to cause the light from the right pixels(224-2) to be focused at the right image plane (or right retina)coinciding the right fovea (106-2). In scenarios in which the left andright images (202-1 and 202-2) are located from the viewer's eyes at adistance comparable to that of the viewer's inter-pupillary distance(the distance between the viewer's two eyes) along the inter-pupillaryline (120), the focal lengths of the left eye (100-1) and the right eye(100-2) may be adjusted by the accommodation process to respectivelydifferent focal lengths.

The accommodation and vergence processes/functions used by the viewer'sbrain in FIG. 2A operate quite differently from the accommodation andvergence processes/functions used by the viewer's brain in FIG. 1B.

For example, as previously noted, in viewing a real world object such asillustrated in FIG. 1B, the viewer's brain uses natural accommodationand vergence processes to control muscles to verge (converge/diverge)toward an object and concurrently (e.g., adapt to) focus on the sameobject. More specifically, the accommodation process in FIG. 1B adjuststhe focal lengths of the left eye lens (108-1) and the right eye lens(108-2) based on the distances between the real world object (114) andthe left and right eye lens (108-1 and 108-2). These distancesself-consistently coincide or terminate/end with the intersection of theoptical axes (102-1 and 102-2) of the left eye (100-1) and the right eye(100-2). In addition, in most cases, as these distances are manymultiples of the inter-pupillary distance, the focal lengths of the lefteye lens (108-1) and the right eye lens (108-2) as set by theaccommodation process in FIG. 1B are substantially the same.

On the other hand, in viewing a virtual object such as illustrated inFIG. 2A, the viewer's brain has to use new and unnatural accommodationand vergence processes to control muscles to verge (converge/diverge)toward the virtual object and concurrently and conflictingly focus onthe pixels (on display(s)) depicting the virtual object rather than thevirtual object at a virtual spatial position. More specifically, theaccommodation process in FIG. 2A adjusts the focal lengths of the lefteye lens (108-1) and the right eye lens (108-2) based on the distancesbetween the left pixels (224-1) and the left eye lens (108-1) andbetween the right pixels (224-2) and the right eye lens (108-2) to fixthe sharpest vision at the display(s) on which the left and right pixels(224-1 and 224-2) are rendered. These distances do not coincide and donot terminate/end with the intersection of the optical axes (102-1 and102-2) of the left eye (100-1) and the right eye (100-2). The left eyelens (108-1) and the right eye lens (108-2), along with the left fovea(106-1) and the right fovea (106), essentially are forced to be directedto two different groups of pixels respectively located at two differentdisplays (or located at different spatial locations of the samedisplay).

In addition, in cases in which the left and right images (202-1 and202-2) are rendered close to the viewer's eyes (100-1 and 100-2), as thedistances between the left pixels (224-1) and the left eye lens (108-1)and between the right pixels (224-2) and the right eye lens (108-2) arecomparable to the inter-pupillary distance, the focal lengths of theleft eye lens (108-1) and the right eye lens (108-2) as set by theaccommodation process in FIG. 2A can be sufficiently different to forcethe viewer's brain to relearn additional new controls of the eyes to usedifferent focal lenses of the eye lenses as a part of overcoming theaccommodation-vergence conflict.

4. Solving Conflict Between Accommodation and Vergence

FIG. 2B illustrates example viewing of the virtual object (214) depictedin the stereoscopic image comprising the left image (252-1) and theright image (252-2) through one or more auto-tunable lenses 228-1,228-2, etc., disposed in between (a) one or more image displays on whichthe left image (252-1) and the right image (252-2) are rendered and (b)the viewer's eyes (100-1 and 100-2). In some embodiments, some or all ofthe image displays are stationary (e.g., at a fixed distance to theviewer, etc.) with respect to the viewer and are parallel (orsubstantially parallel) to the inter-pupillary line (120). For thepurpose of illustration only, the auto-tunable lenses comprise a leftauto-tunable lens 228-1 disposed in between the left image (252-1) andthe left eye (100-1) and a right auto-tunable lens 228-2 disposed inbetween the right image (252-2) and the right eye (100-2). Exampleauto-tunable lenses as described herein may include, but are notnecessarily limited to only, any of: mechanically controllableauto-tunable lenses, electronically controllable auto-tunable lenses,liquid-based auto-tunable lenses (or fluid filled lenses), deformablelenses, Alvarez lenses, etc.

An image processing system as described herein may be used in any of avariety of display applications including but not limited to: 3D displayapplications, multi-view display applications, spherical image displayapplications, virtual reality (VR) applications, augmented reality (AR)applications, remote presence applications, etc. The image processingsystem can be configured to perform gaze tracking operations (and/or eyetracking operations) in real time to determine which specific virtualspatial location is being looked at by the viewer's eyes at any giventime in a display application that renders 3D or multi-view images to beviewed by the viewer. For the purpose of illustration, based on resultsof the gaze/eye tracking operations, the image processing systemdetermines/deduces the specific virtual spatial location at which thevirtual object (214) is located. The gaze/eye tracking operations can bebased on any combination of one or more (e.g., real-time) gaze/eyetracking methods. For example, these gaze/eye tracking methods mayoperate with one or more of: eye attachments, optical sensors, eye imageacquisition and analyses, electric field measurements, infrared light,etc.

In various embodiments, the specific virtual spatial location may berepresented in any spatial coordinate system (e.g., a Cartesiancoordinate system, a polar coordinate system, the World coordinatesystem, a relative coordinate system, etc.) represented in an imagerendering environment in which the viewer is located.

Based on the specific virtual spatial location at the given time, theimage processing system identifies/determines the virtual object plane216 at which the specific virtual spatial location (or the virtualobject (214)) is located at the given time. For example, based on thespecific virtual spatial location at the given time, the imageprocessing system may compute a single depth 236 (or virtual objectdepth) for the virtual object plane (216) at the given time. The singledepth (236) may, but is not necessarily limited to, be represented by adistance between the virtual object plane (216) and the auto-tunablelenses (228-1 and 228-2). Both of the left image (252-1) and the rightimage (252-2) rendered at the image rendering planes may be projectedrespectively by the left auto-tunable lens (228-1) and the rightauto-tunable lens (228-2) to the virtual object plane (216).

In some embodiments, the image processing system uses lens equation(s)to determine focal length(s) of the left and right auto-tunable lenses(228-1 and 228-2).

In a non-limiting implementation example, a single focal length may becomputed using a lens equation; both the left and right auto-tunablelenses (228-1 and 228-2) may be automatically tuned to the same computedsingle focal length. The input to the lens equation in computing thissingle focal length may comprise (a) the single depth (236) representedby the distance between the virtual object plane (216) and theauto-tunable lenses (228-1 and 228-2) and (b) an image display depth(242) represented by a distance between the auto-tunable lenses (228-1and 228-2) and the image displays on which the left image (252-1) andthe right image (252-2) are rendered. The single depth (236) may be usedas the image distance (d2) in the lens equation, whereas the imagedisplay depth (242) may be used as the object distance (d1) in the lensequation. An example of the lens equation to compute a focal length ofan auto-tunable lens as described herein is provided as follows:

$\begin{matrix}{{\frac{1}{d\; 1} + \frac{1}{d\; 2}} = \frac{1}{f}} & (1)\end{matrix}$

The presence of the auto-tunable lenses (228-1 and 228-2) effectivelymoves images from the image displays at which the left image (252-1) andthe right image (252-2) are actually rendered to a virtual image display(represented by the virtual object plane (216)) onto which the leftimage (252-1) and the right image (252-2) are projected by theauto-tunable lenses (228-1 and 228-2).

To effectuate a clear vision of the virtual object (214) through both ofthe left fovea (106-1) of the left eye (100-1) and the right fovea(106-2) of the right eye (100-2), the viewer's brain uses a vergenceprocess (e.g., a divergence process, a convergence process, etc.) tocontrol extraocular muscles in both eyes simultaneously to orient theleft eye (100-1) and the right eye (100-2) toward the virtual object(214).

As a result, the left optical axis (102-1) of the left eye (100-1) andthe right optical axis (102-2) of the right eye (100-2) coincide at thevirtual object (214) to cause light from the virtual object (214) to beprojected onto the left fovea (106-1) of the left eye (100-1) and theright fovea (106-2) of the right eye (100-2). More specifically, thelight from the virtual object (214) comprises (a) a left light portionas projected by the left auto-tunable lens (228-1) from left pixels234-1 (in the left image (252-1)) and (b) a right light portion asprojected by the right auto-tunable lens (228-2) from right pixels 234-2(in the right image (252-2)), where the left pixels (234-1) and theright pixels (234-2) depicts the virtual object (214). The left lightportion corresponding to virtual image portions of the left pixels(234-1) is received by the left fovea (106-1) of the left eye (100-1),whereas the right light portion corresponding to virtual image portionsof the right pixels (234-2) is received by the right fovea (106-2) ofthe right eye (100-2).

At the same time while the vergence process is used to control theextraocular muscles in both eyes simultaneously to orient the left eye(100-1) and the right eye (100-2) toward the virtual object (214), theviewer's brain uses an accommodation process to control ciliary musclesin both eyes simultaneously to focus the left eye (100-1) and the righteye (100-2) respectively onto the virtual object (214) at the virtualobject plane (216). The focal lengths of the left eye (100-1) and theright eye (100-2) may be adjusted by the accommodation process such thatlight from the virtual object (214) is focused at respective imageplanes coinciding the left fovea (106-1) and the right fovea (106-2).More specifically, the focal length of the left eye (100-1) may be setby the accommodation process based at least in part on a (left) distancebetween the virtual spatial location at which the virtual object (214)is located and the left eye lens (108-1) to cause the left light portionof the light from the virtual object (214) to be focused at the leftimage plane coinciding the left fovea (106-1), whereas the focal lengthof the right eye (100-2) may be set by the accommodation process basedat least in part on a (right) distance between the virtual spatiallocation at which the virtual object (214) is located and the right eyelens (108-2) to cause the right light portion of the light from thelight from the virtual object (214) to be focused at the right imageplane coinciding the right fovea (106-2). In scenarios in which thevirtual spatial location at which the virtual object (214) is located isfrom the viewer's eyes at a distance much greater than that of theviewer's inter-pupillary distance (the distance between the viewer's twoeyes) along the inter-pupillary line (120), the focal lengths of theleft eye (100-1) and the right eye (100-2) may be the same orapproximately the same.

The accommodation process used by the viewer's brain in FIG. 2B isidentical or substantially the same as the accommodation process used bythe viewer's brain in FIG. 1B. For example, as in FIG. 1B, theaccommodation process in FIG. 2B adjusts the focal lengths of the lefteye lens (108-1) and the right eye lens (108-2) based on the distancesbetween the virtual object (214) that is in the viewer's foveal visionand the left and right eye lens (108-1 and 108-2). As in FIG. 1B, thesedistances in FIG. 2B self-consistently coincide or terminate/end withthe intersection of the optical axes (102-1 and 102-2) of the left eye(100-1) and the right eye (100-2). In addition, as in FIG. 1B, in mostcases, as these distances are many multiples of the inter-pupillarydistance, the focal lengths of the left eye lens (108-1) and the righteye lens (108-2) as set by the accommodation process in FIG. 1B aresubstantially the same.

Under some approaches, for a single stereoscopic image, multiplerendered images (e.g., 6 rendered images at six different depths, 12rendered images at 12 different depths, rendered images at continuouslyvariable depths, etc.) may be generated and displayed at multipledepths. Objects depicted in the stereoscopic image may be displayed inone of the multiple rendered images based on depths of the depictedobjects in relation to the depths of the multiple rendered images. Thisleads to a display system in which numerous rendered images aregenerated and displayed and in which some of the depicted objects inbetween the multiple depths are represented inaccurately and/or needintensive computations to interpolate the depicted objects that havedepths mismatching the multiple depths of the multiple rendered images.For a given frame refresh rate (e.g., 60 frames per second, 120 framesper second, etc.) of a display system, displaying multiple renderedimages at multiple different depths for a single stereoscopic image isprone to generating perceptible image judders.

In contrast, techniques as described herein can be used to determine asingle depth at which a single stereoscopic or multi-view image shouldbe displayed based on where a viewer is currently looking. This allows adisplay system that implement the techniques as described herein topresent/project a left image and a right image to a single depth or to avery low number of depths, for example, through the use of one or moreauto-tunable lenses. The techniques effectively solve theaccommodation-vergence conflict as accommodation and vergence processescan adapt the viewer's eyes toward the same spatial location at whichimage details of the stereoscopic image that, for example, depict anobject/person. In addition, the techniques as described herein can bedeployed in a wide variety of display systems including but notnecessarily limited to only any of: near-eye displays, head-mounteddisplays, zSpace displays, cinema displays, large display systems,medical display systems, high frame rate display systems, relatively lowframe rate display systems, 3D glasses, TVs, etc.

In some embodiments, an auto-tunable lens (e.g., 228-1, 228-2, etc.)comprises a single lens element; a focal length of the single lenselement can be adjusted based on where an image (e.g., 252-1, 252-2) isto be projected to by the auto-tunable lens.

In some embodiments, an auto-tunable lens (e.g., 228-1, 228-2, etc.)comprises multiple lens elements; some or all of focal lengths of themultiple lens elements can be adjusted based on where an image (e.g.,252-1, 252-2) is to be projected to by the auto-tunable lens. Forexample, one or more lens elements of the auto-tunable lens can bedetermined/selected by an image processing system as described herein toproject a portion of the image to a virtual object plane (216), wherethe portion of the image includes a specific virtual spatial location(on the virtual object plane (216)) that is within the viewer's fovealvision (e.g., 106-1, 106-2, etc.). A distance (e.g., 232-1, 232-2, etc.)between the specific virtual spatial location and the lens elements canbe determined based on geometric relationships (e.g., single depth(236), etc.) between the virtual object plane (216) and a plane (e.g.,locations of the lens elements, the location of the auto-tunable lens,etc.) at which the auto-tunable lens is located. Based at least in parton the distance between the specific virtual spatial location and thelens elements, the focal lengths of the lens elements can be determinedin one or more lens equations.

In some embodiments, a single auto-tunable lens is used to project botha left image and a right image on one or more image displays to avirtual object plane at a single depth at any given time point.

Different viewers may have different vision characteristics includingnear-sightedness, far-sightedness, normal stereoscopic vision, abnormalstereoscopic vision, wearing glasses, wearing no glasses, wearingcontact lenses, etc. Additionally, optionally, or alternatively,different viewers may have different head geometric characteristicsincluding inter-pupillary distances, eye-to-auto-tunable-lens distances,etc. In some embodiments, an image processing system may be capable ofbeing specifically calibrated for a specific viewer with specific visioncharacteristics and/or specific head geometric characteristics. Forexample, before the image processing system is used for viewing 3D ormulti-view images of a display application, in a calibration session, atest stereoscopic image may be presented by the image processing systemto the viewer at different depths distributed around a single depthcorresponding to a reference viewer with a perfect/reference vision. Acorrected depth specific to the viewer may be determined automaticallywith or without user input. This process may be repeated for the viewerover each of a plurality of depths corresponding to the reference viewerwith the perfect/reference vision. A curve, a lookup table (LUT), etc.,may be determined in the calibration session. The viewer's specificdepths may be computed by adjusting runtime computed depths for thereference viewer with the perfect/reference vision. Additionally,optionally, or alternatively, the viewer's specific head geometriccharacteristics may be measured or inputted into the image processingsystem for the purpose of performing accurate gaze/eye tracking andaccurate placement of virtual object planes onto which images atdifferent time points are to be projected to by one or more auto-tunablelenses of the image processing system.

It should be noted that objects/persons including the virtual object(214) depicted in the left image (252-1) and the right image (252-2) maybe magnified by the auto-tunable lenses (228-1 and 228-2). In someembodiments, based at least in part on the single depth (236), the imageprocessing system determines a magnification factor. For example, themagnification factor may be determined as a ratio of the single depth(236) over the image display depth (242). The image processing systemcan perform aspect ratio adjustment operations based on themagnification factor determined in connection with the single depth(236) of the virtual object plane (216). For example, the imageprocessing system may receive or decode, from an input video signal, aninput left image and an input right image that are used to derive theleft image (252-1) and the right image (252-2). The image processingsystem can apply an inverse of the magnification factor to the inputleft image and the input right image to generate the left image (252-1)and the right image (252-2) so that objects/persons—which may or may notbe located at the virtual object plane (216)—depicted in the left image(252-1) and the right image (252-2) as perceived with the presence ofthe auto-tunable lenses (228-1 and 228-2) match in aspect ratio, sizes,etc., with the same objects/persons depicted in the input left image andthe input left image as perceived without the presence of theauto-tunable lenses (228-1 and 228-2).

Depth information in the left image (252-1) and the right image (252-2)as perceived at the image displays without the presence of theauto-tunable lenses (228-1 and 228-2) may be altered into new depthinformation in projected images as perceived at the virtual object plane(216) with the presence of the auto-tunable lenses (228-1 and 228-2).

In some embodiments, depth information can be derived from a combinationof disparity image(s) related to the input left and right images, camerageometric information of a (virtual or real) camera system thatacquires/generates/produces the input left and right images, etc. Insome embodiments, the depth information can be directly read from depthmap(s) that are received with the input left and right images. Some orall of the disparity image(s), the depth map(s), the camera geometricinformation, etc., can be received with the left and right images in theinput video signal.

Additionally, optionally, or alternatively, in some embodiments, theimage processing system applies depth correction operations to the inputleft image and the input right image in generating the left image(252-1) and the right image (252-2) so that new depth information ofobjects/persons—which may or may not be located at the virtual objectplane (216)—depicted in the left image (252-1) and the right image(252-2) as perceived with the presence of the auto-tunable lenses (228-1and 228-2) match input depth information of the same objects/personsdepicted in the input left image and the input left image as perceivedwithout the presence of the auto-tunable lenses (228-1 and 228-2).

Additionally, optionally, or alternatively, in some embodiments, a(e.g., low-strength, etc.) blurring filter with variable spatialresolutions may be applied to image processing operations that generatethe left image (252-1) and the right image (252-2) to decimate tovarying degrees high spatial frequency content from the left image(252-1) and the right image (252-2) that are located outside theviewer's foveal vision. The blurring filter may be used to simulateblurring functions performed by the viewer's retina. The strength ofblurring filtering may increase as the distances of image details asrepresented at the virtual object plane (e.g., 216-1, etc.) relative tothe viewer's foveal vision increase.

In some embodiments, the blurring filter performs no or little blurringin image details that encompass the viewer's foveal vision (e.g., 106-1,106-2, etc.), and performs increasingly stronger blurring in imagedetails that are increasingly away from the viewer's foveal vision, forexample, in areas of retinas (e.g., 110-1, 110-2, etc.) that are outsidethe viewer's foveal vision (e.g., 106-1, 106-2, etc.).

For example, as illustrated in FIG. 2E, a first blurring filter can beused to perform no or little blurring in image details in the left image(252-1) that are projected onto a left foveal vision section 256 of thevirtual object plane (e.g., 216-1, etc.). The left foveal vision section(256) of the virtual object plane encompasses the viewer's left fovealvision (106-1). The first blurring filter can be used to performstronger blurring in image details in the left image (252-1) that areprojected onto one or more left non-foveal vision sections 258 of thevirtual object plane (e.g., 216-1, etc.). The left non-foveal visionsections (258) of the virtual object plane are not in the viewer's leftfoveal vision (106-1).

Likewise, a second blurring filter can be used to perform no or littleblurring in image details in the right image (252-2) that are projectedonto a right foveal vision section (not shown) of the virtual objectplane (e.g., 216-1, etc.). The right foveal vision section of thevirtual object plane encompasses the viewer's right foveal vision(106-2). The second blurring filter can be used to perform strongerblurring in image details in the right image (252-2) that are projectedonto one or more right non-foveal vision sections (not shown) of thevirtual object plane (e.g., 216-1, etc.). The right non-foveal visionsections of the virtual object plane are not in the viewer's rightfoveal vision (106-2).

5. Tracking Vergence Angles

FIG. 2C illustrates example tracking of a viewer's vergence angles inviewing a time sequence of stereoscopic images. The time sequence ofstereoscopic images may depict one or more scenes, a sub-division of ascene, a group of pictures (GOP), etc. Each of the stereoscopic imagesmay be represented by a combination of two 3D images or a combination oftwo or more multi-view images. As used herein, vergence angles refer toviewing angles of an individual eye (e.g., the left eye, the right eye,etc.) of the viewer. Example vergence angles to be tracked by techniquesas described herein may include, but are not necessarily limited toonly, any of: left vergence angles (or vergence angles of the left eye),right vergence angles (or vergence angles of the right eye), vergenceangles relative to a reference direction such as one of the viewer'sinter-pupillary line (120), the viewer's frontal viewing direction(118), etc.; elevation angles relative to the viewer's level viewingplane or the viewer's (vertical) midline of face; etc.

For the purpose of illustration only, at a first time point, an imageprocessing system as described herein determines or measures that theviewer's eyes are looking at the virtual object (214) located at thevirtual object plane (216) at a first stereoscopic image comprising acombination of a first left image and a second right image as projectedby one or more auto-tunable lenses of the image processing system to thevirtual object plane (216) from the image displays at which imagesrepresenting the time sequence of stereoscopic images are rendered.

From the first time point to a second time point, the viewer's eyesconverge or diverge to a second virtual object 214-1 located at a secondvirtual object plane 216-1, depicted in the time sequence ofstereoscopic images.

The second time point may be a time point that is one of: immediatelyfollowing the first time point, following the first time point by one ormore frame time intervals (each frame time interval corresponding todisplaying one image frame), following the first time point by afraction of frame time interval, etc.

As illustrated in FIG. 2C, at the second time point, the viewer's lefteye converges (moving inwardly) to the second virtual object (214-1),whereas the viewer's right eye diverges (moving outwardly) to the secondvirtual object (214-1). It should be noted that in various scenarios,the viewer's eyes can also both converge, or both diverge.

In some embodiments, the image processing system can measure/track theviewer's vergence angles at the second time point. Based on the viewer'svergence angles at the second time point, the image processing systemcan determine that the viewer is looking at the second virtual object(214-1) located at the second virtual object plane (216-1).

In response to determining that the viewer is looking at the secondvirtual object (214-1) located at the second virtual object plane(216-1), the image processing system can project a second stereoscopicimage represented by a combination of a second left image and a secondright image to the second virtual object plane (216-1). The secondstereoscopic image may be a stereoscope image (in the time sequence ofstereoscopic images) that is one of: immediately following the firststereoscopic image, following the first stereoscopic image by one ormore frame time intervals (each frame time interval corresponding todisplaying one image frame), following the first stereoscopic imagewithin a (e.g., strictly) fixed time duration, etc.

FIG. 2D illustrates example viewing of a stereoscopic image comprisingthe left image (252-1) and the right image (252-2) through one or moreauto-tunable lenses 228-1, 228-2, etc., disposed in between (a) one ormore image displays on which the left image (252-1) and the right image(252-2) are rendered and (b) the viewer's eyes (100-1 and 100-2). Asillustrated, the image displays on which the left image (252-1) and theright image (252-2) may be provided by a single display screen 254. Inan example, the single display screen (254) may be a display screen of amobile phone, a personal digital assistant (PDA), an e-book reader, aTV, etc.

In some embodiments, a left image (e.g., 252-1) and a right image (e.g.,252-2) of a stereoscopic image (e.g., an input stereoscopic image, amodified stereoscopic image derived from an input stereoscopic image,etc.) as described herein can be rendered either concurrently or framesequentially in one or more image displays that may or may not overlapwith one another. Additionally, optionally, or alternatively, multipleimage displays as described herein used to render the stereoscopic imagemay be provided by a single display screen or by multiple displayscreens.

In some embodiments, two image displays are used to render the leftimage and the right image of the stereoscopic image, and are provided bytwo distinct display screens (e.g., two display screens in a near-eyedisplay device, etc.) respectively. In these embodiments, the left imageand the right image may be concurrently displayed to the viewer.Additionally, optionally, or alternatively, the left image and the rightimage may be frame sequentially displayed to the viewer with one of theleft image and the right image displayed first and followed by the otherof the left image and the right image displayed.

In some embodiments, two image displays are used to render the leftimage and the right image of the stereoscopic image, and are provided bytwo spatial sections on a single display screen (e.g., two spatialsections on an iPhone screen, etc.) respectively. In some embodiments,these two spatial sections may not be overlapped. In some embodiments,these two spatial sections may be at least partly overlapped. In allthese embodiments, the left image and the right image may beconcurrently displayed to the viewer. Additionally, optionally, oralternatively, the left image and the right image may be framesequentially displayed to the viewer with one of the left image and theright image displayed first and followed by the other of the left imageand the right image displayed.

In some embodiments, one image display is used to render the left imageand the right image of the stereoscopic image, and is provided by asingle display screen (e.g., a TV, etc.). The left image and the rightimage may be concurrently displayed to the viewer. The concurrentlydisplayed images may be distinguished by using different lightwavelengths, different lenticular views, different light polarizations,etc. Additionally, optionally, or alternatively, the left image and theright image may be frame sequentially displayed to the viewer with oneof the left image and the right image displayed first and followed bythe other of the left image and the right image displayed.

In various embodiments, different views (e.g., the left and rightimages, etc.) of a stereoscopic image (or multi-view image) as describedherein may be distinguished in an image processing system from oneanother by one or more of: different display screens, different spatialsections of a single display screen, different frames at different timepoints, different (e.g., non-overlapping) light wavelengths assigned todifferent views, different lenticular views assigned to different views,different light polarizations to different views, etc.

6. Vision Fields

Techniques as described herein can be used to design optical and/orphysical architectures of an eyewear device (e.g., for 3D displayapplications, for virtual reality applications, for augmented realityapplications, for remote presence applications, etc.) that is embeddedwith focus tunable lenses. The eyewear device with the focus tunablelenses may operate in conjunction with a gaze tracker, and provides asolution to the accommodation-vergence conflict problem in viewingobjects rendered on a wide variety of image displays such as near-eyedisplays, head-mounted displays, zSpace displays, cinema displays, largedisplay systems, medical display systems, high frame rate displaysystems, relatively low frame rate display systems, 3D glasses, TVs,etc.

As illustrated in FIG. 2C, a gaze tracker as described herein may trackvergence directions of a viewer's eyes. The gaze tracker may beconfigured to use one or more gaze tracking methods using eyeattachments, optical sensors, eye image acquisition and analyses,electric field measurements, infrared light, etc., Based on the vergenceangles of the viewer's eyes and/or depth information of an objectlocated at the intersection of the left and right gazes of the viewer'seyes, a virtual object depth may be determined for the object.

The focus tunable lenses in the eyewear device may be set to a specificfocal length (or diopter power) within a tunable focal length range toproject left and right images (e.g. 252-1 and 252-2 of FIG. 2B, etc.) ofa 3D or multi-view image depicting the object to the virtual objectdepth that corresponds to the intersection of the left and right gazesof the viewer's eyes, thereby reversing the unnatural eye accommodationat image displays back to natural accommodation at the virtual objectdepth the object should be represented in the 3D or multi-view image.

In some cases, it may be difficult and pointless to build focus tunablelenses that cover complete vision fields of the eyes. Under techniquesas described herein, not all of a vision field of the viewer's left orright eye needs adjustable focus. In these cases, adjustable focus (ortunable focal lengths) may be provided only in specific regions of thevision fields of the eyes. The specific regions of the vision fields ofthe eyes may be significant to the viewer's 3D viewing experiences ascompared with other regions of the vision fields. In some embodiments,outside these specific regions, only non-tunable focal lengths areprovided.

In some embodiments, adjustable focus is provided in specific regions(in a vision field of an eye) such as some or all of: a central region(e.g., fovea, fovea plus a safety margin, etc.), a paracentral region(e.g., excluding and extending from the central region, etc.), anear-peripheral region (e.g., excluding and extending from theparacentral region, etc.), a mid-peripheral region (e.g., excluding andextending from the near peripheral region, etc.), etc. In a non-limitingimplementation, adjustable focus is provided in all of the centralregion, the paracentral region and the near peripheral region.Additionally, optionally or alternatively, the adjustable focus may beat least partly (e.g., 30%, 50%, 80%, etc.) provided in themid-peripheral region.

FIG. 6A illustrates an example vision field of an eye. Each ofconcentric circles (e.g., labelled as 30°, 60°, 90°, etc.) representsdirections of equal (or the same) angular degree relative to an opticalaxis (e.g., 102-1 or 102-2 of FIG. 2B, etc.) of the viewer's left orright eye. The optical axis (e.g., 102-1 or 102-2 of FIG. 2B, etc.)represents a gaze direction (not shown in FIG. 6A) that is pointedvertically out of the plane of FIG. 6A at the intersection of atransverse direction 612 and a vertical direction 614 in the centralregion (602). Here, the transverse direction (612) and the verticaldirection (614) form a plane vertical to the optical axis (e.g., 102-1or 102-2 of FIG. 2B, etc.).

As illustrated in FIG. 6A, the vision field of the eye may be (e.g.,logically, etc.) partitioned into a central region 602 (the darkest fillpattern) immediately surrounded by a paracentral region 604. In someembodiments, the central region (602) may correspond to the viewer'sfovea vision and extend from zero (0) angular degree to a first angle(e.g., 3-7 angular degrees, 5-9 angular degrees, etc.) relative to theoptical axis. The optical axis (e.g., 102-1 or 102-2 of FIG. 2B, etc.)is a direction (not shown in FIG. 6A) that is pointed out at the centerof the central region (602) from, and vertical to, the plane of FIG. 6A.In some embodiments, the paracentral region (604) may extend from thefirst angle to a second angle (e.g., 6-12 angular degrees, etc.)relative to the optical axis (e.g., 102-1 or 102-2 of FIG. 2B, etc.).

The paracentral region (604) is immediately surrounded by anear-peripheral region 606. The near-peripheral region (606) isimmediately adjacent to the mid-peripheral region (608), which in turnis immediately adjacent to the rest of the vision field, afar-peripheral region 310. In some embodiments, the near-peripheralregion (606) may extend from the second angle to a third angle (e.g.,25-35 angular degrees, etc.) relative to the optical axis (e.g., 102-1or 102-2 of FIG. 2B, etc.). In some embodiments, the mid-peripheralregion (608) may extend from the third angle to a fourth angle (e.g.,50-65 angular degrees, etc.) relative to the optical axis (e.g., 102-1or 102-2 of FIG. 2B, etc.). The far-peripheral region (610) may extendfrom the fourth angle to the edge of the vision field.

The first, second, third and fourth angles used in this example logicalpartition of the vision field may be defined or specified along thetransverse direction (612). When the vision field of FIG. 6A correspondsto that at a front level viewing direction, the transverse direction(612) may be the same as, or parallel to, the viewer's inter-pupillaryline 120.

It should be noted that different schemes of logically partitioning aviewer's vision field may be used in addition to, or in place of, thescheme of logically partitioning the viewer's vision field into central,paracentral, near-peripheral, mid-peripheral, far-peripheral, etc.,regions based on angles as illustrated in FIG. 6A.

For example, in some embodiments, the viewer's vision field may bepartitioned into more or fewer regions such as a combination of acentral region, a near-peripheral region and a far-peripheral region,etc., without a paracentral region and/or a mid-peripheral region. Focustunable lens may be used to cover from the central region up to some orall of the near-peripheral region in such logical partition of theviewer's vision field.

In some embodiments, the viewer's vision field may be partitioned basedon other quantities other than angles as previously illustrated. Forexample, in a non-limiting implementation, the central region may bedefined as a vision field portion that corresponds a viewer's fovealvision. The paracentral central region may be defined as a vision fieldportion that corresponds a viewer's retina portion where cone/roddensities exceed relatively high cone/rod density thresholds. Thenear-peripheral region may be defined as a vision field portion thatcorresponds a viewer's retina portion where cone/rod densities does notexceed relatively high cone/rod density thresholds respectively but doesexceed intermediate cone/rod density thresholds. The mid-peripheralregion may be defined as a vision field portion that corresponds aviewer's retina portion where cone/rod densities does not exceedintermediate cone/rod density thresholds respectively but does exceedrelatively low cone/rod density thresholds. Focus tunable lens may beused to cover from the viewer's foveal vision up to some or all of aregion based on threshold(s) (e.g., cone/rod density threshold(s), etc.)that are not necessarily angle-based.

Additionally, optionally or alternatively, a combination of two or moredifferent schemes of logically partitioning the viewer's vision fieldand/or other human vision factors may be used to determine how theviewer's vision field should be covered by a focus tunable lens. Forexample, instead of using a focus tunable lens to cover the same angularvalue range in different angular directions, the focus tunable lens maycover a larger angular value range along the transverse direction (612)than an angular value range covered by the same focus tunable lens alongthe vertical direction (614), as the human vision system may be moresensitive to image details along the transverse direction (612) thanthose along the vertical direction (614).

FIG. 6B and FIG. 6C illustrate example (top-view) angular ranges 618-1and 618-2 in some or all of which adjustable focus may be provided by aneyewear device implementing techniques as described herein. A viewer'sgaze direction (or angle), along with the viewer's fovea vision 616, mayswivel or rotate around, not necessarily only horizontally or onlyvertically, from time to time, for example, in an angular value rangebetween 45 angular degrees left and right to the left of the viewer'sfrontal viewing direction (118).

In some embodiments, adjustable focus is provided to cover some or allof: a central region (e.g., fovea, fovea plus a safety margin, etc.), aparacentral region (e.g., excluding and extending from the centralregion, etc.), a near-peripheral region (e.g., excluding and extendingfrom the paracentral region, etc.), a mid-peripheral region (e.g.,excluding and extending from the near peripheral region, etc.), etc.,for the entire angular value range in which they may swivel or rotatearound.

In some embodiments, adjustable focus is provided to cover some or allof: a central region (e.g., fovea, fovea plus a safety margin, etc.), aparacentral region (e.g., excluding and extending from the centralregion, etc.), a near-peripheral region (e.g., excluding and extendingfrom the paracentral region, etc.), a mid-peripheral region (e.g.,excluding and extending from the near peripheral region, etc.), etc.,for only a portion of the entire angular range in which the viewer'sgaze may swivel or rotate around.

In some embodiments as illustrated in FIG. 6B, adjustable focus isprovided to cover a symmetric angular range 618-1 representing asymmetric (to the front viewing direction (118)) portion of the wideangular range. Examples of the symmetric angular range (618-1) mayinclude, but are not necessarily limited to, one of: +/−15 angulardegrees, +/−20 angular degrees, +/−25 angular degrees, etc., relative tothe viewer's frontal viewing direction (118).

In some embodiments as illustrated in FIG. 6C and FIG. 6D, adjustablefocus is provided to cover an asymmetric angular range (e.g., 618-2,618-3, etc.) representing an asymmetric (to the front viewing direction(118)) portion of the wide angular range. An asymmetric angular range ina vision field of one eye may be defined or specified as covering froman interior angle (looking towards the other/conjugate eye) to anexterior angle (looking away from the other/conjugate eye).

In the embodiments as illustrated in FIG. 6C, the asymmetric angularrange (618-2) is biased with a preference towards interior directionsoverlapped in both vision fields of the viewer's left and right eyes.Examples of the interior angle of the asymmetric angular range (618-2)with bias to interior angles may include, but are not necessarilylimited to, one of: 15 angular degrees, 30 angular degrees, 45 angulardegrees, etc., relative to the viewer's frontal viewing direction (118).Examples of the exterior angle of the asymmetric angular range (618-2)with bias to interior angles may include, but are not necessarilylimited to, one of: 10 angular degrees, 15 angular degrees, 20 angulardegrees, etc., relative to the viewer's frontal viewing direction (118).

In the embodiments as illustrated in FIG. 6D, the asymmetric angularrange (618-3) is biased with a preference towards exterior directionswhich may or may not be overlapped in both vision fields of the viewer'sleft and right eyes. Examples of the exterior angle of the asymmetricangular range (618-3) with bias to exterior directions may include, butare not necessarily limited to, one of: 15 angular degrees, 30 angulardegrees, 45 angular degrees, etc., relative to the viewer's frontalviewing direction (118). Examples of the interior angle of theasymmetric angular range (618-3) with bias to exterior directions mayinclude, but are not necessarily limited to, one of: 10 angular degrees,15 angular degrees, 20 angular degrees, etc., relative to the viewer'sfrontal viewing direction (118).

FIG. 6E illustrates another example vision field 620 of an eye. Ascompared with the example vision field of FIG. 6A, the vision field(620) of FIG. 6E takes into consideration vision-related factors such aseye swiveling, viewing constraints from nose, corneal, eyelid, etc. Eachof concentric circles (e.g., labelled as 15°, 60°, 90°, etc.) in FIG. 6Erepresents directions of equal (or the same) angular degree relative toan optical axis (e.g., 102-1 or 102-2 of FIG. 2B, etc.) of the viewer'sleft or right eye. The optical axis (e.g., 102-1 or 102-2 of FIG. 2B,etc.) represents the viewer's frontal viewing direction (e.g., 118 ofFIG. 2B) that is pointed vertically out of the plane of FIG. 6E at theintersection of a transverse direction 622 and a vertical direction 624.Here, the transverse direction (622) and the vertical direction (624)form a plane vertical to the frontal viewing direction (e.g., 118 ofFIG. 2B).

In some embodiments, adjustable focus is provided to cover the entirevision field (620). In some embodiments, adjustable focus is provided tocover a specific region 626 in the vision field (620) rather than theentire vision field (620). The specific region (626) may be either aregular shape or an irregular shape. Examples of the specific region(626) may include, but are not necessarily limited to, any combinationof one or more of: circular shapes (e.g., as illustrated in FIG. 8Athrough FIG. 8E, etc.), oblong shapes, oval shapes, heart shapes, starshapes, round shapes, square shapes, etc.

7. Example Focus Tunable Lenses

FIG. 7A illustrates an example (cross-sectional-view) liquid-based focustunable lens 700-1 that may be incorporated into an eyewear device asdescribed herein. By way of example but not limitation, the liquid-basedfocus tunable lens (700-1) may comprise one or more curved membranes702-1, 702-2, 702-3, etc. Liquid may be filled or emptied out byactuators, etc., into one or more volumes 704-1, 704-2, 704-3, etc., tomaintain or reshape the curvature of each of the curved membranes (702).Example membranes as described herein may include, without limitation,transparent membranes, elastically expandable membranes, etc. Exampleliquid as described herein may include, without limitation, transparentfluid, optically uniform fluid, high-optical-transparency fluid withrelatively high refractive index and/or with relatively low viscosity,fluid with low optical distortions or impurities, etc.

At any given time in operation, an eyewear device as described hereinthat incorporates the liquid-based focus tunable lens (700-1) maydetermine specific focal length(s) to which the liquid-based focustunable lens (700-1) is to be tuned (e.g., in order to project astereoscopic image to a virtual object depth as determined by gazetracking, etc.). Based on the specific focal length(s), the eyeweardevice controls (through one or more of mechanical devices,electromechanical devices, piezoelectric devices, electro-fluidicdevices, digital control devices, etc.) amount(s) of the liquid filledinto or emptied out from the volumes to reshape one, some or all of thecurvatures of the curved membranes (702) to produce the specific focallength(s) (for focusing incoming light substantially from opposite to aviewing direction 708) in specific regions of the liquid-based focustunable lens (700-1). For example, the curvature of the membrane (702-1)may be controlled by filling liquid in or emptying liquid out from thevolumes (704-1 through 704-3); the curvature of the membrane (702-2) maybe controlled by filling liquid in or emptying liquid out from thevolumes (704-2 and 704-3); the curvature of the membrane (702-3) may becontrolled by filling liquid in or emptying liquid out from the volume(704-3). As a result, a first spatial region 706-1 in the liquid-basedfocus tunable lens (700-1) is set to a first focal length in thespecific focal length(s); a second spatial region 706-2 in theliquid-based focus tunable lens (700-1) is set to a second focal lengthin the specific focal length(s); a third spatial region 706-3 in theliquid-based focus tunable lens (700-1) is set to a third focal lengthin the specific focal length(s). In some embodiments, one or more of thespecific focal length(s) (e.g., the first focal length in the firstspatial region (706-1)) may be used to project images of a 3D ormulti-view image depicting an object to a virtual object depth at whichthe object is to be located in the 3D or multi-view image.

FIG. 7B illustrates an example (front-view) liquid-crystal based focustunable lens 700-2 that may be incorporated into an eyewear device asdescribed herein. The liquid-crystal based focus tunable lens (700-2)may comprise one or more liquid crystal layers, switching elements,electrodes, substrates, polarizers, matrixes, etc. The liquid-crystalbased focus tunable lens (700-2), or the liquid crystal layers therein,may be partitioned into multiple individually controllable spatialportions, such as columns, rows, cells, pixilation units, stripes, etc.,distributed (e.g., a two-dimensional array, etc.) over a curved orplanar surface such as a combination of one or more of: circular shapes,oblong shapes, oval shapes, heart shapes, star shapes, round shapes,square shapes, etc. Example liquid crystal materials used in liquidcrystal layers as described herein may include, without limitation,birefringence liquid crystal materials, non-birefringence liquid crystalmaterials, etc.

In some embodiments, an eyewear device, or a focus tunable lens and/orswitching elements therein, may be operatively and/or communicativelycoupled to other devices such as one or more of: image displays, eyegaze tracking devices, video streaming clients, video streaming servers,auto-tunable lens controllers, etc., for the purpose of adjusting/tuningfocal lengths of the eyewear device in runtime.

By way of example but not limitation, the liquid-crystal based focustunable lens (700-2) comprises pixilation units represented by aplurality of liquid crystal cells distributed over a rectangularsurface. The refractive index of each liquid crystal cell in theplurality of liquid crystal cells may be individually (e.g., gradually,collaboratively, etc.) controllable, for example by electric fieldsgenerated by setting specific voltage signal (voltage level and/orphase) with electrodes and switching elements. Example electrodes usedto generate electric fields as described herein may include, withoutlimitation, transparent conductive thin film, conductive stripes, etc.Example switch elements used to control driving voltages of electrodesas described herein may include, without limitation, transistors, etc.

At any given time in operation, an eyewear device as described hereinthat incorporates the liquid-crystal based focus tunable lens (700-2)may determine specific focal length(s) to which the liquid-crystal basedfocus tunable lens (700-2) is to be tuned. Based on the specific focallength(s), the eyewear device controls (through one or more of electricdevices, electro-optical devices, digital control devices, etc.)refractive indexes of some or all of the liquid crystal cells to producefocusing effects that correspond to the specific focal length(s) (forfocusing incoming light from opposite to a viewing direction that issubstantially perpendicular to the plane of FIG. 7B) in specific regionsof the liquid-crystal based focus tunable lens (700-2). In an example,refractive index(es) of all of the liquid crystal cells in theliquid-crystal based focus tunable lens (700-2) may be (e.g.,individually, variably, collectively, collaboratively, etc.) controlledto produce light bending or light focusing effects that correspond tothe specific focal length(s). In another example, refractive indexes ofliquid crystal cells in a subset (e.g., a specific spatial region 710,etc.) of the liquid crystal cells in the liquid-crystal based focustunable lens (700-2) may be (e.g., individually, variably, collectively,collaboratively, etc.) controlled to produce light bending or lightfocusing effects that correspond to the specific focal length(s). Insome embodiments, one or more of the specific focal length(s) may beused to project images of a 3D or multi-view image depicting an objectto a virtual object depth at which the object is to be located in the 3Dor multi-view image.

It should be noted that focus tunable lens as described herein may beimplemented with other technologies, besides liquid-based tunable lensand liquid-crystal based tunable lens. For example, a focus tunable lensmay be mechanically implemented with two or more movable self-adjustablelenses under the Alvarez principle. Depending on these movable lensesoverlapped with each other, different focal lengths may be generated indifferent regions of the movable lenses. Based on eye gaze tracking,these lenses may be moved relative to one another to tune/adjust one ormore specific regions of the movable lenses to specific focal lengths.Furthermore, the one or more specific regions with the specific focallengths may be positioned to cover central regions (e.g., foveal visionplus a safety margin, etc.) of a viewer's vision field(s).

8. Example Eyewear Devices

FIG. 8A illustrates an example left lens assembly 802-1 of an eyeweardevice (e.g., 800 of FIG. 8B, FIG. 8C or FIG. 8E, etc.) that implementsan optical and/or physical architecture under techniques as describedherein. FIG. 8B through FIG. 8F illustrate example perspective, frontand top views of the eyewear device (800).

The eyewear device (800) comprises the left lens assembly (802-1) to beplaced in front of the left eye of a viewer (808 of FIG. 8D or FIG. 8F)and a right lens assembly (e.g., 802-2 of FIG. 8B, FIG. 8C or FIG. 8E,etc.) in front of the right eye of the viewer (808). In someembodiments, the left lens assembly (802-1) has a left focus tunablelens 804 and a left focus fixed lens 806. Similarly, the right lensassembly (802-2) has a right focus tunable lens and a right focus fixedlens.

In some embodiments, a focus tunable lens as described herein covers allof a near-peripheral region (e.g., 606 of FIG. 6A, etc.) in a visionfield of the left or right eye of the viewer (808). For example, theleft focus tunable lens (804) may completely cover a leftnear-peripheral region in a left vision field of the left eye of theviewer (808). Similarly, the right focus tunable lens may completelycover a right near-peripheral region in a right vision field of theright eye of the viewer (808). A vision field (e.g., FIG. 6A, FIG. 6B,FIG. 6C, etc.) as described herein may be specified or defined inreference to a specific frontal viewing direction (which may be aspecific eye gaze direction of the left or right eye, a specificvergence angle of the left or right eye, etc.) of the viewer (808).Additionally, optionally or alternatively, the vision field may bespecified or defined in reference to a specific front viewingdirectional range (e.g., an angular range of eye swivel, a centralportion of an angular range of eye swivel, etc.).

In some embodiments, the left or right near-peripheral region (e.g., 606of FIG. 6A, etc.) represents an angular range along a horizontaldirection (e.g., the transverse direction (612) of FIG. 6A, etc.)relative to one of: an optical axis, the specific front viewingdirection, a central direction in the specific front viewing directionalrange, etc., of the left or right eye of the viewer (808). Exampleangular ranges of the near-peripheral region (606) may include, withoutlimitation, one of: up to 15 degrees, greater than 15 degrees but nogreater than 30 degrees, greater than 30 degrees but no greater than 60degrees, greater than 60 degrees, etc.

In some embodiments, the left focus tunable lens (804) at leastpartially covers a left mid-peripheral region (e.g., 608 of FIG. 6A,etc.) of the left eye of the viewer (808). Similarly, the right focustunable lens at least partially covers a right mid-peripheral region(e.g., 608 of FIG. 6A, etc.) of the right eye of the viewer (808).

In some embodiments, the left or right mid-peripheral region (e.g., 608of FIG. 6A, etc.) represents an angular range along a horizontaldirection (e.g., the transverse direction (612) of FIG. 6A, etc.)relative to one of: an optical axis, the specific front viewingdirection, a central direction in the specific front viewing directionalrange, etc., of the left or right eye of the viewer (808). The angularrange of the left or right mid-peripheral region (e.g., 608 of FIG. 6A,etc.) may be larger than and adjacent to the angular range of the leftor right near-peripheral region (e.g., 606 of FIG. 6A, etc.).

In some embodiments, the eyewear device (800), or an auto-tunable lenscontroller (e.g., 318 of FIG. 3B, etc.) operating in conjunction withthe eyewear device (800), controls one or more focal lengths of the leftfocus tunable lens (804) in the left lens assembly (802-1) to project aleft image of a stereoscopic image to a virtual object plane at avirtual object depth. The one or more focal lengths of the left focustunable lens (804) are determined based at least in part on the virtualobject depth (which may in turn be determined by vergence angles of theleft and right eyes of the viewer (808) through eye gaze tracking).

Similarly, the eyewear device (800) or the auto-tunable lens controller(318) controls one or more focal lengths of the right focus tunable lensin the right lens assembly (802-2) to project a right image of thestereoscopic image to the virtual object plane at the virtual objectdepth. The one or more focal lengths of the right focus tunable lens aredetermined based at least in part on the virtual object depth.

In some embodiments, a focus tunable lens as described herein maysupport a single tunable focal length, for example, in an entireliquid-based focus tunable lens, in the first spatial region (706-1) inthe liquid-based focus tunable lens (700-1) of FIG. 7A, in an entireliquid-crystal based focus tunable lens (e.g., 700-2 of FIG. 7B), in aspatial region (710) of the liquid-crystal based focus tunable lens(e.g., 700-2 of FIG. 7B), etc. In some other embodiments, a focustunable lens as described herein may support multiple tunable focallengths, for example, in different spatial regions (e.g., 706-1, 706-2,706-3, etc.) in the liquid-based focus tunable lens (700-1) of FIG. 7A,in different spatial regions (e.g., 710, outside 710, etc.) of theliquid-crystal based focus tunable lens (e.g., 700-2 of FIG. 7B), etc.

By way of example but not limitation, in some embodiments, the one ormore focal lengths of the left focus tunable lens (804) or the rightfocus tunable lens may comprise one or more of: (a) a first focal lengththat corresponds to a central region (e.g., 602 of FIG. 6A, etc.) of theleft or right near-peripheral region (e.g., 606 of FIG. 6A, etc.); (b) asecond focal length that corresponds to a paracentral region (e.g., 604of FIG. 6A, etc.) of the left or right near-peripheral region excludingthe central region (e.g., 602 of FIG. 6A, etc.); (c) a third focallength that corresponds to the left or right near-peripheral region(e.g., 606 of FIG. 6A, etc.) excluding the central region (e.g., 602 ofFIG. 6A, etc.) and the paracentral region (e.g., 604 of FIG. 6A, etc.);(d) a fourth focal length that corresponds to a mid-peripheral region(e.g., 608 of FIG. 6A, etc.) excluding the central region (e.g., 602 ofFIG. 6A, etc.), the paracentral region (e.g., 604 of FIG. 6A, etc.) andthe near-peripheral region (e.g., 606 of FIG. 6A, etc.); etc. Thesemultiple focal lengths may be individually or collectively controlled.In some embodiments, the central region (e.g., 602 of FIG. 6A, etc.)corresponds to the viewer's foveal vision. In some embodiments, thecentral region (e.g., 602 of FIG. 6A, etc.) covers at least a non-fovealportion of the vision field of the left or right eye of the viewer(808).

In some embodiments, all of the multiple tunable focal lengths are setto the same focal length in projecting the left or right image of thestereoscopic image.

In some embodiments, some or all of the multiple tunable focal lengthsare set to at least two different focal lengths in projecting the leftor right image of the stereoscopic image. In some embodiments, a (e.g.,tunable or adjustable in runtime, etc.) focal length in a central regionof the viewer's vision field the first focal length is adjustable in agreater focal length range than a focal length range in which otherfocal length(s) are adjusted. In the previous example, the first focallength of the central region may be adjustable in a larger focal lengthrange (e.g., twice, 20% more, 30% more, etc.) than a focal length rangein which the fourth focal length in the mid-peripheral region (e.g., 608of FIG. 6A, etc.) excluding the central region (e.g., 602 of FIG. 6A,etc.), the paracentral region (e.g., 604 of FIG. 6A, etc.) and thenear-peripheral region (e.g., 606 of FIG. 6A, etc.) is adjustable. Insome embodiments, the closer a region of a vision field is to thecentral region, the larger a focal length range for tuning/adjusting afocal length in the region is.

In some embodiments, the focus tunable lens such as the left focustunable lens (804), the right focus tunable lens, etc., comprises one ormore adjustable focal lengths in a plurality of spatial regions of thefocus tunable lens (e.g., 804, etc.). Additionally, optionally oralternatively, the one or more adjustable focal lengths in the pluralityof spatial regions of the focus tunable lens (e.g., 804, etc.) form arelatively smooth, a relatively gradual, a monotonicallydescending/ascending, etc., transition of focal lengths from a centralspatial region in the plurality of spatial regions to a boundary spatialregion in the plurality of spatial regions. In some embodiments, thecentral spatial region corresponds to a central region of thenear-peripheral region of a vision field of the left or right eye of theviewer (808), whereas the boundary spatial region is spatiallyimmediately adjacent to the focus fixed lens such as the left focusfixed lens (806), the right focus fixed lens, etc.

In some embodiments, an eye lens assembly (e.g., 802-1, etc.), or afocus tunable lens (e.g., 804, etc.) and a focus fixed lens (e.g., 806,etc.) therein, form a curved structure that substantially surrounds theleft or right eye of the viewer (808). For example, the curved structureformed by the focus tunable lens (e.g., 804, etc.) and the focus fixedlens (e.g., 806, etc.) in the eye lens assembly (e.g., 802-1, etc.) maysubstantially (e.g., over 80%, 90%, etc.) cover the entire vision field,completely cover the entire vision field, completely cover the entirevision field plus a safety margin (e.g., a 5% safety margin, a 10%safety margin, etc.). The lenses in the eye lens assembly maycollectively be curvilinear following a portion of the viewer's headcontour. The portion of the viewer's head contour may include some orall of the viewer's eye areas, fringe facial areas around the viewer'seye areas, parts of the viewer's temple areas, etc.

For the purpose of illustration, it has been described that an eyeweardevice (800) as described herein can be used to operate with imagedisplays that display stereoscope images or multi-view images so that aviewer (e.g., 808 of FIG. 8D or FIG. 8F) can view these images in a true3D experience without experiencing the accommodation-vergence conflictsas commonly or frequently occurred under other approaches that do notimplement techniques as described herein. Thus, the eyewear device (800)can operate in a dynamic operational mode such as dynamically tuningfocus tunable lenses in the eyewear device (800) to different focallengths based on which objects are being looked at by the viewer (808)in the stereoscope images or multi-view images at any given time in a 3Ddisplay application, VR application, AR application, etc.

It should be noted that in various embodiments, an eyewear device (800)as described herein may also be used in non-dynamic operational modes inwhich focus tunable lens in the eyewear device may be set to specificfocal lengths that are fixed or relatively slow changing. For example,the eyewear device (800) may be used as a prescription glass that is setto compensate the viewer's specific near-sightedness, far-sightedness,progressive (e.g., a combination of near-sightedness for far objects andfar-sightedness for close objects, etc.), etc., for example, as measuredby an optician, as adjusted by the viewer (808), etc. Thus, the viewer(808) can wear the eyewear device (800) to view real life objects (e.g.,reading a book, etc.) or real life scenes in a real life environment,rather than in an artificial environment depicted/rendered instereoscope images or multi-view images.

Additionally, optionally or alternatively, even when an eyewear deviceas described herein is used for viewing objects depicted in stereoscopeimages or multi-view images, the eyewear device (800), or anauto-tunable lens controller (e.g., 318 of FIG. 3A, FIG. 3B or FIG. 3C,etc.) operating in conjunction with the eyewear device (800), may takeindividual vision characteristics (e.g., specific near-sightedness,far-sightedness, progressive, etc.) of a viewer (e.g., 800 of FIG. 3D orFIG. 3F, etc.) into consideration, and project the objects to theirrespective virtual object depths in a manner that compensate for theviewer's individual vision characteristics. For example, anadjusted/tuned focal length of a focus tunable lens as described hereinmay include a first portion that compensate for the near-sightedness orfar-sightedness of the viewer (808) and a second portion that projectsthe objects to their respective virtual object depths.

In some embodiments, a focus tunable lens (e.g., 804 of FIG. 8A, etc.)as described herein may be configured to cover a larger or smaller(e.g., within +/−5 angular degrees, within +/−10 angular degrees etc.)vision field portion as compared with a standard vision viewer, if theviewer (808) is determined to have a larger or smaller clear visionfield portion as compared with such standard vision viewer.

A viewer's visual perception characteristics are different in differentportions of the viewer's vision field. For example, the viewer'speripheral vision may not be sensitive to color differences and imageresolution as compared with the viewer's central vision. In someembodiments, a focus fixed lens (e.g., 806 of FIG. 8A, etc.) that coversthe viewer's peripheral vision outside vision field portions covered bythe focus tunable lens (e.g., 804 of FIG. 8A, etc.) may be configured toprovide peripheral image information sensitive to the viewer'speripheral vision such as peripheral object structures, peripheralobject movements, peripheral object visual characteristic changes (e.g.,flashing, size changing, etc.). In some embodiments, different portionsof the focus fixed lens (e.g., 806 of FIG. 8A, etc.) may be fixed to asingle fixed focal length. In some embodiments, different portions ofthe focus fixed lens (e.g., 806 of FIG. 8A, etc.) may be fixed tomultiple fixed focal lengths. At runtime (e.g., in a 3D displayapplication, in a VR application, in an AR application, etc.), the fixedfocal length(s) of the focus fixed lens (e.g., 806 of FIG. 8A, etc.) arefixed independent of any objects to which the viewer (808) directs eyegazes. In some embodiments, through the different fixed focal lengths,the focus fixed lens (e.g., 806 of FIG. 8A, etc.) may be configured toprovide a perception of rectilinear space in the viewer's peripheralvision with no or little spatial distortions.

In some embodiments, a focus tunable lens (e.g., 804 of FIG. 8A, etc.)may or may not have the same spatial dimension/size along differentspatial directions. In some embodiments, as illustrated in FIG. 8A, thefocus tunable lens (804) may have a first spatial dimension/size(denoted as “y”) along the vertical direction, and have a second spatialdimension/size (denoted as “x”) along the horizontal direction. In someembodiments, the first and second spatial dimensions/sizes may be thesame or similar. In some embodiments, the first and second spatialdimensions/sized may be different. For example, subject toimplementation details, at a distance to the eye of 3-20 millimeters,the first spatial dimension/size (“y”) may have a range of 30 to 45millimeters, whereas the second spatial dimension/size (“x”) may have arange of 45 to 60 millimeters. A horizontal spatial dimension/size(denoted as “c”) of the focus fixed lens (806) may range between 10millimeters and 60 millimeters at a distance to the viewer's headcontour of 3-20 millimeters.

Focus fixed lenses (e.g., 806 of FIG. 8A, etc.) and focus tunable lenses(e.g., 804 of FIG. 8A, etc.) in an eyewear device as described hereinmay or may not be implemented by the same lens technologies. In someembodiments, a focus tunable lens (e.g., 804 of FIG. 8A, etc.) and afocus fixed lens (e.g., 806 of FIG. 8A, etc.) outside the focus tunablelens (e.g., 804 of FIG. 8A, etc.) may be implemented using the same lenstechnologies. In some embodiments, a focus tunable lens (e.g., 804 ofFIG. 8A, etc.) may be implemented using a first lens technology, whereasa focus fixed lens (e.g., 806 of FIG. 8A, etc.) outside the focustunable lens (e.g., 804 of FIG. 8A, etc.) may be implemented using asecond different lens technology.

Additionally, optionally or alternatively, a focus tunable lens (e.g.,804 of FIG. 8A, etc.) and a focus fixed lens (e.g., 806 of FIG. 8A,etc.) outside the focus tunable lens (e.g., 804 of FIG. 8A, etc.) mayimplement a smooth transition of focal lengths to avoid sharp fall-offor sudden changes in focal lengths.

Example shapes of the focus tunable lens (804) may include, but are notnecessarily limited to, any combination of one or more of: circularshapes, oblong shapes, oval shapes, heart shapes, star shapes, roundshapes, square shapes, etc.

A spatial configuration such as one or more of spatial location, spatialshape, spatial dimension, etc., of a focus tunable lens (e.g., 804 ofFIG. 8A, etc.) as described herein may or may not vary in relation toother parts (e.g., a focus fixed lens, etc.) of an eyewear device (e.g.,800, etc.).

In some embodiments, the spatial configuration of a focus tunable lens(e.g., 804 of FIG. 8A, etc.) is fixed in relation to other parts of aneyewear device (e.g., 800, etc.). The focus tunable lens (e.g., 804 ofFIG. 8A, etc.) may be located in a fixed part of an eye lens assembly(e.g., 802-1, etc.) with a fixed spatial shape in relation to a focusfixed lens (e.g., 806 of FIG. 8A, etc.) outside the focus tunable lens(e.g., 804 of FIG. 8A, etc.), even though focal length(s) of the focustunable lens (e.g., 804 of FIG. 8A, etc.) are (e.g., automatically,etc.) adjustable/tunable.

In some other embodiments, the spatial configuration of a focus tunablelens (e.g., 804 of FIG. 8A, etc.) is not fixed in relation to otherparts of an eyewear device (e.g., 800, etc.). The focus tunable lens(e.g., 804 of FIG. 8A, etc.) may be located at a first spatial locationof an eye lens assembly (e.g., 802-1, etc.) at a first time point. Thefocus tunable lens (e.g., 804 of FIG. 8A, etc.) may be located at asecond different spatial location of the eye lens assembly (e.g., 802-1,etc.) at a second different time point. For example, based on gazetracking, at the first time, the central region of the left or right eyeof the viewer (808) may be at or near the first spatial location; at thesecond time, the central region of the left or right eye of the viewer(808) may be at or near the second spatial location. Thus, at the firsttime, the focus tunable lens (e.g., 804 of FIG. 8A, etc.), which may beformed by movable fluid-filled lenses, formed by activating/controllingrefractive indexes of pixilation units of a liquid-crystal based focustunable lens, etc., may be set to cover the first spatial location.Similarly, at the second time, the focus tunable lens (e.g., 804 of FIG.8A, etc.) may be set to cover the second spatial location.

9. Example Video Streaming Servers and Clients

FIG. 3A illustrates an example video streaming server 300 that comprisesan image processor 302, a depth-based image generator 312, etc. In someembodiments, the image processor (302) comprises an image receiver 306,a data repository 310, etc. Some or all of the components of the videostreaming server (300) may be implemented by one or more devices,modules, units, etc., in software, hardware, a combination of softwareand hardware, etc.

In some embodiments, the image receiver (306) comprises software,hardware, a combination of software and hardware, etc., configured toreceive an input image stream 304 from an image source such as acloud-based image source, a camera system in connection with a VRapplication, an AR application, a remote presence application, a 3Ddisplay application, a multi-view display application, etc.; decode theinput image stream (304) into one or more input stereoscopic images(e.g., a sequence of input stereoscopic images, a sequence of inputmulti-view images, etc.); etc. In some embodiments, the input imagestream (304) may carry image metadata (e.g., camera geometricinformation, etc.) that can be decoded by the image receiver (306) fromthe input image stream (304).

In some embodiments, the data repository (310) represents one or moredatabases, one or more data storage units/modules/devices, etc.,configured to support operations such as storing, updating, retrieving,deleting, etc., with respect to some or all of the input stereoscopicimages, the image metadata, etc. In some embodiments, the inputstereoscopic images are retrieved from the data repository (310) by thedepth-based image generator (308) instead of the input image stream(304).

In some embodiments, the depth-based image generator (308) comprisessoftware, hardware, a combination of software and hardware, etc.,configured to receive, via a bidirectional data flow 314, vergenceangles, etc., of a user (or a viewer) over time; generate an outputvideo stream comprising depth-based stereoscopic images and depthcontrol metadata; provide/transmit the output video stream via thebidirectional data flow 314 directly or indirectly through intermediatedevices, etc.) to a stereoscopic video streaming client, a displaydevice, a storage device, etc. In various embodiments, the depth-basedstereoscopic images may refer to input stereoscopic images (e.g.,received by the video streaming server (300), etc.) or modifiedstereoscopic images derived from input stereoscopic images.

The depth control metadata may represent a function of single depthsover time. A single depth at any given time point as indicated in thedepth control metadata may be determined computed based at least in parton the vergence angles of the user and other geometric information(e.g., distances between image displays and the user's eyes, distancesbetween image displays and auto-tunable lenses, etc.) related to imagedisplays, etc., and may be used to control one or more focal lengths ofone or more auto-tunable lenses in an image rendering device that isbeing used by the user to view the stereoscopic images to project acorresponding stereoscopic image for which the single depth isdetermined/computed.

Additionally, optionally, or alternatively, some or all of imageprocessing operations such as aspect ratio adjustment operations, depthcorrection operations, blurring filtering, scene cut detections,transformations between coordinate systems, temporal dampening, displaymanagement, content mapping, color mapping, field-of-view management,etc., may be performed by the stereoscopic video streaming server (300)for the purpose of generating the depth-based stereoscopic images andthe depth control metadata encoded into the output video stream.

The video streaming server (300) may be used to support real time visionapplications, near-real-time vision applications, non-real-time visionapplications, virtual reality, augmented reality, remote presence,helmet mounted display applications, heads up display applications,games, 2D display applications, 3D display applications, multi-viewdisplay applications, etc.

FIG. 3B illustrates an example image rendering system 324-1 thatcomprises a depth-based image receiver 316, a vergence angle tracker326, an auto-tunable lens controller 318, one or more image displays320, etc. Some or all of the components of the image rendering system(324-1) may be implemented by one or more devices, modules, units, etc.,in software, hardware, a combination of software and hardware, etc.

A user (or viewer) may move the user's vergence angles to image details(or visual objects/persons depicted in the stereoscopic images) ofdifferent depths respectively at different time points at runtime. Insome embodiments, the vergence angle tracker (326) comprises software,hardware, a combination of software and hardware, etc., configured totrack vergence angles, etc., of the user over time. The user's vergenceangles over time may be sampled or measured at a relatively fine timescales (e.g., every millisecond, every five milliseconds, etc.).

In some embodiments, the depth-based image receiver (316) comprisessoftware, hardware, a combination of software and hardware, etc.,configured to send, via a bidirectional data flow 314, the user'svergence angles, other geometric information (e.g., distances betweenimage displays and the user's eyes, distances between image displays andauto-tunable lenses, etc.), etc.; receive a video stream (e.g.,outputted by an upstream device, etc.) comprising depth-basedstereoscopic images and depth control metadata corresponding to thedepth-based stereoscopic images; etc.

The image rendering system (324-1) is configured to decode the receivedvideo stream into the depth-based stereoscopic images and the depthcontrol metadata; render the depth-based stereoscopic images (e.g., eachof which comprising left and right images), as decoded from the receivedvideo stream, on the image displays (320).

In some embodiments, the auto-tunable lens controller (318) comprisessoftware, hardware, a combination of software and hardware, etc.,configured to use the depth control metadata to control one or moreauto-tunable lenses to project the stereoscopic images, as rendered onthe image displays (320), to virtual images at the different depthsrespectively at the different time points (e.g., subject to a real timeprocessing delay of a millisecond, 10 millisecond, a fraction of a frametime, etc.).

Additionally, optionally, or alternatively, some or all of imagerendering operations such as gaze/eye tracking, aspect ratio adjustmentoperations, depth correction operations, blurring filtering, scene cutdetections, temporal dampening of time-varying image parameters, anyother temporal manipulation of image parameters, display management,content mapping, tone mapping, color mapping, field-of-view management,prediction, navigations through mouse, trackball, keyboard, foottracker, actual body motion, etc., may be performed by the imagerendering system (324-1).

The image rendering system (324-1) may be used to support real time,near real time, or non-real time vision applications, near-real-timevision applications, non-real-time vision applications, virtual reality,augmented reality, remote presence, helmet mounted display applications,heads up display applications, games, 2D display applications, 3Ddisplay applications, multi-view display applications, etc.

Techniques as described herein can be implemented in a variety of systemarchitectures. Some or all image processing operations as describedherein can be implemented by any combination of one or more ofcloud-based video streaming servers, video streaming servers collocatedwith or incorporated into video streaming clients, image renderingsystems, image rendering systems, display devices, etc. Based on one ormore factors such as types of vision applications, bandwidth/bitratebudgets, computing capabilities, resources, loads, etc., of recipientdevices, computing capabilities, resources, loads, etc., of videostreaming servers and/or computer networks, etc., some image processingoperations can be performed by a video streaming server, while someother image processing operations can be performed by a video streamingclient, an image rendering system, a display device, etc.

FIG. 3C illustrates an example configuration in which a depth-basedimage generator (e.g., 312, etc.) is incorporated into an edge videostreaming server 324-2. In some embodiments, an image processor 302 ofFIG. 3C may be cloud-based. In some embodiments, the image processor(302) may be located in a core network separate from edge devices suchas the edge video streaming server (324-2). As in FIG. 3A, the imageprocessor (302) may comprise an image receiver 306, a data repository310, etc. The image processor (302) may represent an upstream videostreaming server that communicates with the edge video streaming server(324-2) over relatively high bitrates. Some or all of the components ofthe image processor (302) and/or the edge video streaming server (324-2)may be implemented by one or more devices, modules, units, etc., insoftware, hardware, a combination of software and hardware, etc.

In some embodiments, the image processor (302) is configured to sendinput stereoscopic images in a data flow 322 to downstream devices oneof which may be the edge video streaming server (324-2).

In some embodiments, the edge video streaming server (324-2), or thedepth-based image generator (312) therein, comprises software, hardware,a combination of software and hardware, etc., configured to determinevergence angles, other geometric information, etc., of a user over time;generate an output video stream comprising depth-based stereoscopicimages and depth control metadata; provide/transmit the output videostream via the bidirectional data flow 314 directly or indirectlythrough intermediate devices, etc.) to a video streaming client, adisplay device, a storage device, etc.

10. Example Process Flows

FIG. 4A illustrates an example process flow according to an exampleembodiment of the present invention. In some example embodiments, one ormore computing devices or components may perform this process flow. Inblock 402, an image processing system (e.g., any combination of a videostreaming server or a video streaming client of FIG. 3A through FIG. 3C,etc.) determines, while a viewer is viewing a first stereoscopic imagecomprising a first left image and a first right image, a left vergenceangle of a left eye of a viewer and a right vergence angle of a righteye of the viewer.

In block 404, the image processing system determines, based at least inpart on (i) the left vergence angle of the left eye of the viewer and(ii) the right vergence angle of the right eye of the viewer, a virtualobject depth.

In block 406, the image processing system renders a second stereoscopicimage comprising a second left image and a second right image for theviewer on one or more image displays. The second stereoscopic image issubsequent to the first stereoscopic image.

In block 408, the image processing system projects the secondstereoscopic image from the one or more image displays to a virtualobject plane at the virtual object depth.

FIG. 4B illustrates another example process flow according to an exampleembodiment of the present invention. In some example embodiments, one ormore computing devices or components may perform this process flow. Inblock 422, an image processing system (e.g., any combination of aneyewear device, a video streaming server or a video streaming client ofFIG. 3A through FIG. 3C, etc.) uses one or more gaze tracking devices totrack a virtual object depth to which a viewer's left eye and theviewer's right eye are directed.

In block 424, the image processing system renders a stereoscopic imagecomprising a left image and a right image on one or more image displays.

In block 426, the image processing system projects the left image to avirtual object plane at a virtual object depth with a left lens assemblyof an eyewear device.

In block 428, the image processing system projects the right image tothe virtual object plane at the virtual object depth with a right lensassembly of the eyewear device.

In an embodiment, the left lens assembly comprises a left focus tunablelens and a left focus fixed lens, whereas the right lens assemblycomprises a right focus tunable lens and a right focus fixed lens.

In an embodiment, the second left image and the second right image aregenerated from an input left image and an input right image by applyingone or more aspect ratio adjustment operations that adjust aspect ratiosof visual objects depicted in the input left image and the input rightimage to modified aspect ratios of the visual objects depicted in thesecond left image and the second right image as projected to the virtualobject plane at the virtual object depth.

In an embodiment, the second left image and the second right image aregenerated from an input left image and an input right image by applyingone or more depth correction operations that converts depths of visualobjects depicted in the input left image and the input right image tomodified depths of the visual objects depicted in the second left imageand the second right image as projected to the virtual object plane atthe virtual object depth.

In an embodiment, the second left image and the second right image aregenerated from an input left image and an input right image by applyingone or more blurring filtering operations that blur the one or morespatial regions of the second left image and the second right imagerelative to the input left image and the input right image; the one ormore spatial regions of the second left image and the second right imageare away from the viewer's foveal vision.

In an embodiment, the second left image and the second right image ofthe second stereoscopic image are rendered concurrently to be viewed bythe viewer.

In an embodiment, the second left image and the second right image ofthe second stereoscopic image are rendered frame sequentially to beviewed by the viewer.

In an embodiment, the second stereoscopic image is immediatelysubsequent to the first stereoscopic image in time in a sequence ofstereoscopic images that are rendered to be viewed by the viewer.

In an embodiment, one or more auto-tunable lenses are used to projectthe second stereoscopic image to the virtual object plane at the virtualobject depth; one or more focal lengths of the one or more auto-tunablelenses are determined based at least in part on the virtual objectdepth.

In an embodiment, the second stereoscopic image is generated from asecond input stereoscopic image in a sequence of input stereoscopicimages; no other stereoscopic image other than the second stereoscopicimage is generated from the second input stereoscopic image; no othervirtual object depth other than the virtual object depth is the secondstereoscopic image projected.

In an embodiment, the second left image and the second right image ofthe second stereoscopic image are rendered on a first image display anda second image display respectively.

In an embodiment, the second left image and the second right image ofthe second stereoscopic image are rendered on a single image display.

In an embodiment, at least one of the second left image and the secondright image of the second stereoscopic image is projected to the virtualobject plane based on an auto-tunable lens comprising a single lenselement.

In an embodiment, at least one of the second left image and the secondright image of the second stereoscopic image is projected to the virtualobject plane based on an auto-tunable lens comprising multiple lenselements.

In an embodiment, the second left image of the second stereoscopic imageis viewable only by the left eye of the viewer, whereas the second rightimage of the second stereoscopic image is viewable only by the right eyeof the viewer.

In an embodiment, the image processing system is further configured toperform: while the viewer is viewing the second stereoscopic imagecomprising the second left image and the second right image, determininga second left vergence angle of the left eye of the viewer and a secondright vergence angle of the right eye of the viewer; determining, basedat least in part on (i) the second left vergence angle of the left eyeof the viewer and (ii) the second right vergence angle of the right eyeof the viewer, a second virtual object depth; rendering a thirdstereoscopic image comprising a third left image and a third right imagefor the viewer on the one or more image displays, the third stereoscopicimage being subsequent to the second stereoscopic image; projecting thethird stereoscopic image from the on one or more image displays to asecond virtual object plane at the second virtual object depth; etc.

In an embodiment, the virtual object depth is adjusted based at least inpart on specific vision characteristics of the viewer.

In various example embodiments, an apparatus, a system, an apparatus, orone or more other computing devices performs any or a part of theforegoing methods as described. In an embodiment, a non-transitorycomputer readable storage medium stores software instructions, whichwhen executed by one or more processors cause performance of a method asdescribed herein.

Note that, although separate embodiments are discussed herein, anycombination of embodiments and/or partial embodiments discussed hereinmay be combined to form further embodiments.

11. Implementation Mechanisms—Hardware Overview

According to one embodiment, the techniques described herein areimplemented by one or more special-purpose computing devices. Thespecial-purpose computing devices may be hard-wired to perform thetechniques, or may include digital electronic devices such as one ormore application-specific integrated circuits (ASICs) or fieldprogrammable gate arrays (FPGAs) that are persistently programmed toperform the techniques, or may include one or more general purposehardware processors programmed to perform the techniques pursuant toprogram instructions in firmware, memory, other storage, or acombination. Such special-purpose computing devices may also combinecustom hard-wired logic, ASICs, or FPGAs with custom programming toaccomplish the techniques. The special-purpose computing devices may bedesktop computer systems, portable computer systems, handheld devices,networking devices or any other device that incorporates hard-wiredand/or program logic to implement the techniques.

For example, FIG. 5 is a block diagram that illustrates a computersystem 500 upon which an example embodiment of the invention may beimplemented. Computer system 500 includes a bus 502 or othercommunication mechanism for communicating information, and a hardwareprocessor 504 coupled with bus 502 for processing information. Hardwareprocessor 504 may be, for example, a general purpose microprocessor.

Computer system 500 also includes a main memory 506, such as a randomaccess memory (RAM) or other dynamic storage device, coupled to bus 502for storing information and instructions to be executed by processor504. Main memory 506 also may be used for storing temporary variables orother intermediate information during execution of instructions to beexecuted by processor 504. Such instructions, when stored innon-transitory storage media accessible to processor 504, rendercomputer system 500 into a special-purpose machine that is customized toperform the operations specified in the instructions.

Computer system 500 further includes a read only memory (ROM) 508 orother static storage device coupled to bus 502 for storing staticinformation and instructions for processor 504.

A storage device 510, such as a magnetic disk or optical disk, solidstate RAM, is provided and coupled to bus 502 for storing informationand instructions.

Computer system 500 may be coupled via bus 502 to a display 512, such asa liquid crystal display, for displaying information to a computer user.An input device 514, including alphanumeric and other keys, is coupledto bus 502 for communicating information and command selections toprocessor 504. Another type of user input device is cursor control 516,such as a mouse, a trackball, or cursor direction keys for communicatingdirection information and command selections to processor 504 and forcontrolling cursor movement on display 512. This input device typicallyhas two degrees of freedom in two axes, a first axis (e.g., x) and asecond axis (e.g., y), that allows the device to specify positions in aplane.

Computer system 500 may implement the techniques described herein usingcustomized hard-wired logic, one or more ASICs or FPGAs, firmware and/orprogram logic which in combination with the computer system causes orprograms computer system 500 to be a special-purpose machine. Accordingto one embodiment, the techniques herein are performed by computersystem 500 in response to processor 504 executing one or more sequencesof one or more instructions contained in main memory 506. Suchinstructions may be read into main memory 506 from another storagemedium, such as storage device 510. Execution of the sequences ofinstructions contained in main memory 506 causes processor 504 toperform the process steps described herein. In alternative embodiments,hard-wired circuitry may be used in place of or in combination withsoftware instructions.

The term “storage media” as used herein refers to any non-transitorymedia that store data and/or instructions that cause a machine tooperation in a specific fashion. Such storage media may comprisenon-volatile media and/or volatile media. Non-volatile media includes,for example, optical or magnetic disks, such as storage device 510.Volatile media includes dynamic memory, such as main memory 506. Commonforms of storage media include, for example, a floppy disk, a flexibledisk, hard disk, solid state drive, magnetic tape, or any other magneticdata storage medium, a CD-ROM, any other optical data storage medium,any physical medium with patterns of holes, a RAM, a PROM, and EPROM, aFLASH-EPROM, NVRAM, any other memory chip or cartridge.

Storage media is distinct from but may be used in conjunction withtransmission media. Transmission media participates in transferringinformation between storage media. For example, transmission mediaincludes coaxial cables, copper wire and fiber optics, including thewires that comprise bus 502. Transmission media can also take the formof acoustic or light waves, such as those generated during radio-waveand infra-red data communications.

Various forms of media may be involved in carrying one or more sequencesof one or more instructions to processor 504 for execution. For example,the instructions may initially be carried on a magnetic disk or solidstate drive of a remote computer. The remote computer can load theinstructions into its dynamic memory and send the instructions over atelephone line using a modem. A modem local to computer system 500 canreceive the data on the telephone line and use an infra-red transmitterto convert the data to an infra-red signal. An infra-red detector canreceive the data carried in the infra-red signal and appropriatecircuitry can place the data on bus 502. Bus 502 carries the data tomain memory 506, from which processor 504 retrieves and executes theinstructions. The instructions received by main memory 506 mayoptionally be stored on storage device 510 either before or afterexecution by processor 504.

Computer system 500 also includes a communication interface 518 coupledto bus 502. Communication interface 518 provides a two-way datacommunication coupling to a network link 520 that is connected to alocal network 522. For example, communication interface 518 may be anintegrated services digital network (ISDN) card, cable modem, satellitemodem, or a modem to provide a data communication connection to acorresponding type of telephone line. As another example, communicationinterface 518 may be a local area network (LAN) card to provide a datacommunication connection to a compatible LAN. Wireless links may also beimplemented. In any such implementation, communication interface 518sends and receives electrical, electromagnetic or optical signals thatcarry digital data streams representing various types of information.

Network link 520 typically provides data communication through one ormore networks to other data devices. For example, network link 520 mayprovide a connection through local network 522 to a host computer 524 orto data equipment operated by an Internet Service Provider (ISP) 526.ISP 526 in turn provides data communication services through the worldwide packet data communication network now commonly referred to as the“Internet” 528. Local network 522 and Internet 528 both use electrical,electromagnetic or optical signals that carry digital data streams. Thesignals through the various networks and the signals on network link 520and through communication interface 518, which carry the digital data toand from computer system 500, are example forms of transmission media.

Computer system 500 can send messages and receive data, includingprogram code, through the network(s), network link 520 and communicationinterface 518. In the Internet example, a server 530 might transmit arequested code for an application program through Internet 528, ISP 526,local network 522 and communication interface 518.

The received code may be executed by processor 504 as it is received,and/or stored in storage device 510, or other non-volatile storage forlater execution.

12. Equivalents, Extensions, Alternatives and Miscellaneous

In the foregoing specification, example embodiments of the inventionhave been described with reference to numerous specific details that mayvary from implementation to implementation. Thus, the sole and exclusiveindicator of what is the invention, and is intended by the applicants tobe the invention, is the set of claims that issue from this application,in the specific form in which such claims issue, including anysubsequent correction. Any definitions expressly set forth herein forterms contained in such claims shall govern the meaning of such terms asused in the claims. Hence, no limitation, element, property, feature,advantage or attribute that is not expressly recited in a claim shouldlimit the scope of such claim in any way. The specification and drawingsare, accordingly, to be regarded in an illustrative rather than arestrictive sense.

What is claimed is:
 1. An eyewear device, comprising: a left lensassembly including: a left focus tunable lens covering a first centralportion of a left vision field of a viewer of the eyewear device; a leftfocus fixed lens covering a first boundary portion of the left visionfield of the viewer of the eyewear device, wherein the first boundaryportion is uncovered by the left focus tunable lens and outside thefirst central portion; a right lens assembly including: a right focustunable lens covering a second central portion of a right vision fieldof the viewer of the eyewear device; a right focus fixed lens covering asecond boundary portion of the right vision field of the viewer of theeyewear device, wherein the second boundary portion is uncovered by theright focus tunable lens and outside the second central portion.
 2. Theeyewear device of claim 1, wherein the left focus tunable lenscompletely covers a left near-peripheral vision of the viewer's left eyein reference to a frontal viewing direction of the viewer.
 3. Theeyewear device of claim 2, wherein the left near-peripheral visionrepresents an angular range along a horizontal direction relative to anoptical axis of the viewer's left eye; and wherein the angular rangerepresents one of: up to 15 degrees, greater than 15 degrees but nogreater than 30 degrees, greater than 30 degrees but no greater than 60degrees, or greater than 60 degrees.
 4. The eyewear device of claim 1,wherein the left focus tunable lens partially covers a leftmid-peripheral vision of the viewer's left eye in reference to a frontalviewing direction of the viewer; and wherein the left mid-peripheralvision encompasses the left near-peripheral vision.
 5. The eyeweardevice of claim 1, wherein the left lens assembly is configured toproject a left image of a stereoscopic image to a virtual object planeat a virtual object depth; and wherein one or more focal lengths of theleft focus tunable lens are determined based at least in part on thevirtual object depth.
 6. The eyewear device of claim 5, wherein the oneor more focal lengths of the left focus tunable lens comprise one ormore of: (a) a first focal length that corresponds to a central regionof the left near-peripheral vision, (b) a second focal length thatcorresponds to a paracentral region of the left near-peripheral visionexclusive of the central region, (c) a third focal length thatcorresponds to a near-peripheral portion of the left near-peripheralvision exclusive of the central region and the paracentral region, or(d) a fourth focal length that corresponds to a mid-peripheral region ofthe left near-peripheral vision exclusive of the central region, theparacentral region, and the near-peripheral region.
 7. The eyeweardevice of claim 6, wherein the central region corresponds to theviewer's foveal vision.
 8. The eyewear device of claim 6, wherein themid-peripheral region corresponds to a portion of the viewer'snon-foveal vision.
 9. The eyewear device of claim 6, wherein the firstfocal length is adjustable in a greater focal length range than a focallength range in which the fourth focal length is adjustable.
 10. Theeyewear device of claim 5, wherein the right lens assembly is configuredto project a right image of the stereoscopic image to the virtual objectplane at the virtual object depth; and wherein one or more focal lengthsof the right focus tunable lens are determined based at least in part onthe virtual object depth.
 11. The eyewear device of claim 1, wherein theleft focus tunable lens comprises one or more adjustable focal lengthsin a plurality of spatial regions of the left focus tunable lens; andwherein the one or more adjustable focal lengths in the plurality ofspatial regions of the left focus tunable lens form a smooth transitionof focal lengths from a central spatial region in the plurality ofspatial regions to a boundary spatial region in the plurality of spatialregions; wherein the central spatial region corresponds to a centralportion of the left near-peripheral vision; and wherein the boundaryspatial region is spatially immediately adjacent to the left focus fixedlens.
 12. The eyewear device of claim 1, wherein the left focus tunablelens and the left focus fixed lens form a curved structure that coversover 80% of the left vision field of the viewer's left eye.
 13. Theeyewear device of claim 1, wherein the eyewear device is a part of avision device comprising: one or more image displays that display a leftimage and a right image of a stereoscopic image; wherein the eyeweardevice projects the left image and the right image to a virtual objectdepth depending on the viewer's vergence angles.
 14. The eyewear deviceof claim 13, wherein the vision device further comprises one or moregaze tracking devices that track and determine the viewer's vergenceangles at runtime.
 15. A method comprising: using one or more gazetracking devices to track a virtual object depth to which a viewer'sleft eye and the viewer's right eye are directed; rendering astereoscopic image comprising a left image and a right image on one ormore image displays; projecting the left image to a virtual object planeat a virtual object depth with a left lens assembly of an eyeweardevice; projecting the right image to the virtual object plane at thevirtual object depth with a right lens assembly of the eyewear device;wherein the left lens assembly comprises a left focus tunable lens and aleft focus fixed lens; wherein the left focus tunable lens covers afirst central portion of a left vision field of the viewer; wherein theleft focus fixed lens covers a first boundary portion of the left visionfield of the viewer of the eyewear device, wherein the first boundaryportion is uncovered by the left focus tunable lens and outside thefirst central portion; wherein the right lens assembly comprises a rightfocus tunable lens and a right focus fixed lens; wherein the right focustunable lens covers a second central portion of a right vision field ofthe viewer; wherein the right focus fixed lens covers a second boundaryportion of the right vision field of the viewer of the eyewear device,wherein the second boundary portion is uncovered by the right focustunable lens and outside the second central portion.
 16. The method ofclaim 15, wherein the left image and the right image are renderedconcurrently to be viewed by the viewer.
 17. The method of claim 15,wherein the left image and the right image are rendered framesequentially to be viewed by the viewer.
 18. The method of claim 15,wherein the left image and the right image are rendered on a first imagedisplay of the image displays and a second different image display ofthe image displays respectively.
 19. The method of claim 15, wherein theleft image and the right image are rendered on a single image display.20. The method of claim 15, wherein the left focus tunable lenscomprises a single lens element for a central region of the leftnear-peripheral vision.
 21. The method of claim 15, wherein the leftimage is visually perceivable only by the viewer's left eye, whereas theright image is visually perceivable only by the viewer's right eye. 22.The method of claim 15, wherein the virtual object depth is adjustedbased at least in part on the viewer's specific vision characteristics.23. An apparatus performing the method as recited in claim
 15. 24. Asystem performing the method as recited in claim
 15. 25. Anon-transitory computer readable storage medium, storing softwareinstructions, which when executed by one or more processors causeperformance of the method recited in claim
 15. 26. A computing devicecomprising one or more processors and one or more storage media, storinga set of instructions, which when executed by one or more processorscause performance of the method recited in claim 15.